09/01/2025 01:58:21.012 7fd783f1b0c0 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/01/2025 01:58:21.012 7fd783f1b0c0 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/01/2025 01:58:21.012 7fd783f1b0c0 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/01/2025 01:58:21.020 7fd7811584a0 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 09/01/2025 01:58:21.020 7fd7811584a0 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 09/01/2025 01:58:21.021 7fd781158860 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 09/01/2025 01:58:21.022 7fd781158860 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:58:21.022 7fd781158860 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/01/2025 01:58:21.022 7fd781158860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/01/2025 01:58:21.022 7fd781158860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.022 7fd781158860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:58:21.022 7fd781158860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.022 7fd781158860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.023 7fd781158860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/01/2025 01:58:21.023 7fd781158860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.156 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 19201094144497964409871908134733399505143473534068017011949581818925834486514, 'start': '1756691898000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 103814123520090509851445463849869939316395940916377114265952578016372100628138, 'start': '1756691900000000', 'statename': 'PROCESS_STATE_BACKOFF'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 37001084163899275511883337664813226694056084662494138768742811764473810521620, 'start': '1756691899000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 09/01/2025 01:58:21.235 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.236 7fd7811b0a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = package contrail-lib is not installed installed_package_version = package contrail-lib is not installed >> 09/01/2025 01:58:21.236 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.236 7fd783f1b0c0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3992-1.', _node_type='Database', _instance_id='0', _timestamp=1756691901236607, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:58:21.236 7fd783f1b0c0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3992-1.', _node_type='Database', _instance_id='0', _timestamp=1756691901236926, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691898000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_BACKOFF', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691900000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691899000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:58:21.239 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.239 7fd7811b0a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:58:21.239 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.239 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.240 7fd7811b0a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691898000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_BACKOFF start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691900000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691899000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 09/01/2025 01:58:21.240 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.518 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 01:58:21.907 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 09/01/2025 01:58:21.910 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:21.910 7fd781158e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 16848608 disk_space_available_1k = 64241876 analytics_db_size_1k = 48 >>, ] >> 09/01/2025 01:58:21.910 7fd781158e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:22.142 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 09/01/2025 01:58:22.142 7fd783f1b0c0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3992-1.', _node_type='Database', _instance_id='0', _timestamp=1756691902142782, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:58:22.144 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:22.144 7fd781158e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:58:22.144 7fd781158e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:23.055 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 01:58:24.059 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7200 repair -pr ContrailAnalyticsCql'. Error is: Result: 1 09/01/2025 01:58:25.023 7fd7811cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:25.027 7fd7811cd540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:25.028 7fd7811cd540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:25.028 7fd7811cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:25.028 7fd7811cd540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:25.028 7fd7811cd540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:25.028 7fd7811cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:25.029 7fd7811cd540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:25.029 7fd7811cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:25.029 7fd7811cd540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:25.029 7fd7811cd540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:25.029 7fd7811cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:25.029 7fd7811cd540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:25.029 7fd7811cd540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:27.096 7fd783f1b0c0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3992-1.', _node_type='Database', _instance_id='0', _timestamp=1756691907095314, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691898000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_BACKOFF', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691900000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691899000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:58:27.097 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16988468, partition_space_available_1k=64102016, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1270548, cpu_share=0.0, mem_res=1260732, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52068, cpu_share=0.0, mem_res=42640, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=7410812, free=7617272, buffers=664736, cached=16972288, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.14, five_min_avg=2.51, fifteen_min_avg=1.8, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 01:58:27.098 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:27.098 7fd7811b0a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691898000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_BACKOFF start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691900000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691899000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 09/01/2025 01:58:27.099 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:27.099 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:27.100 7fd7811b0a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 16988468 partition_space_available_1k = 64102016 percentage_partition_space_used = 21 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1270548 cpu_share = 0.0 mem_res = 1260732 >>contrail-database-nodemgr : << mem_virt = 52068 cpu_share = 0.0 mem_res = 42640 >> } system_mem_usage = << total = 32665108 used = 7410812 free = 7617272 buffers = 664736 cached = 16972288 node_type = database-node >> system_cpu_usage = << one_min_avg = 3.14 five_min_avg = 2.51 fifteen_min_avg = 1.8 cpu_share = 0 node_type = database-node >> >> 09/01/2025 01:58:27.100 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:29.029 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:29.031 7fd7811b0a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:29.032 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:29.032 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:29.032 7fd7811b0a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:29.033 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:29.033 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:29.034 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:29.034 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:29.034 7fd7811b0a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:29.035 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:29.035 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:29.035 7fd7811b0a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:29.035 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:33.033 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:33.034 7fd7811b0a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:33.035 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:33.035 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:33.035 7fd7811b0a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:33.035 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:33.035 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:33.035 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:33.036 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:33.036 7fd7811b0a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:33.036 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:33.036 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:33.036 7fd7811b0a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:33.036 7fd7811b0a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:37.036 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:37.037 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:37.037 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:37.037 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:37.038 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:37.038 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:37.038 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:37.038 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:37.038 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:37.038 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:37.039 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:37.039 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:37.039 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:37.039 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:41.038 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:41.040 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:41.040 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:41.041 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:41.041 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:41.041 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:41.042 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:41.042 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:41.043 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:41.043 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:41.043 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:41.044 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:41.044 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:41.044 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:45.042 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:45.044 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:45.044 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:45.045 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:45.045 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:45.045 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:45.045 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:45.046 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:45.047 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:45.047 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:45.047 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:45.048 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:45.048 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:45.048 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:49.045 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:49.047 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:49.047 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:49.047 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:49.047 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:49.047 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:49.047 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:49.048 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:49.048 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:49.048 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:49.048 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:49.048 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:49.048 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:49.048 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:53.049 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:53.050 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:53.050 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:53.051 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:53.051 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:53.051 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:53.051 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:53.052 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:53.052 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:53.052 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:53.052 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:53.052 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:53.053 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:53.053 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:57.054 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:57.056 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:57.056 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:57.057 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:57.057 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:57.057 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:57.058 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:57.058 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:57.059 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:57.059 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:57.060 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:57.060 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:57.060 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:57.060 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:01.056 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:01.057 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:01.058 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:01.058 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:01.058 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:01.058 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:01.058 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:01.058 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:01.058 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:01.058 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:01.059 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:01.059 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:01.059 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:01.059 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:05.061 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:05.066 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:05.067 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:05.067 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:05.068 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:05.068 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:05.069 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:05.070 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:05.071 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:05.071 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:05.072 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:05.073 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:05.074 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:05.074 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:09.068 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:09.070 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:09.071 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:09.071 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:09.071 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:09.072 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:09.072 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:09.073 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:09.073 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:09.073 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:09.074 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:09.074 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:09.074 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:09.074 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:13.075 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:13.077 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:13.078 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:13.078 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:13.078 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:13.079 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:13.079 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:13.080 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:13.081 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:13.081 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:13.081 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:13.081 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:13.082 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:13.082 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:17.081 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:17.083 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:17.083 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:17.083 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:17.083 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:17.083 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:17.083 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:17.083 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:17.083 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:17.084 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:17.084 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:17.084 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:17.084 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:17.084 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:21.085 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:21.087 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:21.087 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:21.088 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:21.088 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:21.088 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:21.088 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:21.089 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:21.089 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:21.089 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:21.090 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:21.090 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:21.090 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:21.090 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:21.545 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 01:59:22.020 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:22.020 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 17843076 disk_space_available_1k = 63247408 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 01:59:22.020 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:22.267 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 01:59:22.268 7fd783f1b0c0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3992-1.', _node_type='Database', _instance_id='0', _timestamp=1756691962268205, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.254:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:59:22.270 7fd7811b04a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:22.271 7fd7811b04a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:22.271 7fd7811b04a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:23.244 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 01:59:25.089 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:25.094 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:25.094 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:25.094 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:25.095 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:25.095 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:25.096 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:25.097 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:25.097 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:25.098 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:25.098 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:25.098 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:25.099 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:25.099 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:26.296 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17843104, partition_space_available_1k=63247380, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1305180, cpu_share=0.1, mem_res=1294856, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52336, cpu_share=0.02, mem_res=42796, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=7699644, free=6498456, buffers=675140, cached=17791868, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.67, five_min_avg=2.77, fifteen_min_avg=1.94, cpu_share=3.62, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 01:59:26.296 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:26.299 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 17843104 partition_space_available_1k = 63247380 percentage_partition_space_used = 22 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1305180 cpu_share = 0.1 mem_res = 1294856 >>contrail-database-nodemgr : << mem_virt = 52336 cpu_share = 0.02 mem_res = 42796 >> } system_mem_usage = << total = 32665108 used = 7699644 free = 6498456 buffers = 675140 cached = 17791868 node_type = database-node >> system_cpu_usage = << one_min_avg = 3.67 five_min_avg = 2.77 fifteen_min_avg = 1.94 cpu_share = 3.62 node_type = database-node >> >> 09/01/2025 01:59:26.299 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:29.095 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:29.096 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:29.097 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:29.097 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:29.097 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:29.097 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:29.097 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:29.097 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:29.098 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:29.098 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:29.098 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:29.098 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:29.098 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:29.098 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:33.097 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:33.099 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:33.099 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:33.099 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:33.100 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:33.100 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:33.100 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:33.101 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:33.101 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:33.101 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:33.101 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:33.101 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:33.101 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:33.102 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:37.102 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:37.104 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:37.104 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:37.104 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:37.105 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:37.105 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:37.105 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:37.105 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:37.105 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:37.105 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:37.105 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:37.105 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:37.106 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:37.106 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:41.106 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:41.108 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:41.108 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:41.108 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:41.109 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:41.109 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:41.109 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:41.110 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:41.110 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:41.110 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:41.110 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:41.110 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:41.111 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:41.111 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:45.109 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:45.111 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:45.111 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:45.111 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:45.111 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:45.111 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:45.112 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:45.112 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:45.112 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:45.112 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:45.112 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:45.112 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:45.112 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:45.113 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:49.114 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:49.116 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:49.117 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:49.117 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:49.118 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:49.119 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:49.119 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:49.120 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:49.120 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:49.121 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:49.121 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:49.122 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:49.122 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:49.123 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:53.117 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:53.118 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:53.118 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:53.118 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:53.118 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:53.118 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:53.118 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:53.119 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:53.119 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:53.119 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:53.119 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:53.119 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:53.119 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:53.119 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:57.122 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:57.123 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:57.124 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:57.124 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:57.124 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:57.125 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:57.125 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:57.126 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:57.126 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:57.126 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:57.126 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:57.127 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:57.127 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:57.127 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:01.126 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:01.129 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:01.129 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:01.131 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:01.131 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:01.131 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:01.131 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:01.132 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:01.132 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:01.133 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:01.133 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:01.133 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:01.133 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:01.134 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:05.131 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:05.133 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:05.134 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:05.134 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:05.135 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:05.135 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:05.135 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:05.136 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:05.136 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:05.137 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:05.137 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:05.138 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:05.138 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:05.138 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:09.135 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:09.137 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:09.137 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:09.137 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:09.137 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:09.137 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:09.137 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:09.138 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:09.138 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:09.138 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:09.138 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:09.139 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:09.139 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:09.139 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:13.137 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:13.139 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:13.139 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:13.139 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:13.140 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:13.140 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:13.140 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:13.141 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:13.141 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:13.141 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:13.141 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:13.141 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:13.141 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:13.142 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:17.141 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:17.144 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:17.144 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:17.144 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:17.144 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:17.145 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:17.145 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:17.146 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:17.146 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:17.146 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:17.146 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:17.147 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:17.147 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:17.147 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:21.145 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:21.147 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:21.147 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:21.148 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:21.148 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:21.148 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:21.149 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:21.149 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:21.150 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:21.150 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:21.150 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:21.151 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:21.151 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:21.151 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:21.679 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:00:22.241 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:22.248 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19341684 disk_space_available_1k = 61748800 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:00:22.249 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:22.530 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:00:23.704 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:00:25.148 7fd7811b0ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:25.151 7fd7811b0ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:25.152 7fd7811b0ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:25.152 7fd7811b0ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:25.153 7fd7811b0ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:25.153 7fd7811b0ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:25.153 7fd7811b0ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:25.154 7fd7811b0ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:25.154 7fd7811b0ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:25.155 7fd7811b0ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:25.155 7fd7811b0ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:25.155 7fd7811b0ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:25.156 7fd7811b0ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:25.156 7fd7811b0ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:26.747 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19341860, partition_space_available_1k=61748624, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1303800, cpu_share=0.05, mem_res=1293788, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52416, cpu_share=0.03, mem_res=42796, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=8795328, free=3773484, buffers=707348, cached=19388948, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.34, five_min_avg=2.89, fifteen_min_avg=2.03, cpu_share=4.12, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:00:26.747 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:26.748 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19341860 partition_space_available_1k = 61748624 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1303800 cpu_share = 0.05 mem_res = 1293788 >>contrail-database-nodemgr : << mem_virt = 52416 cpu_share = 0.03 mem_res = 42796 >> } system_mem_usage = << total = 32665108 used = 8795328 free = 3773484 buffers = 707348 cached = 19388948 node_type = database-node >> system_cpu_usage = << one_min_avg = 3.34 five_min_avg = 2.89 fifteen_min_avg = 2.03 cpu_share = 4.12 node_type = database-node >> >> 09/01/2025 02:00:26.748 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:29.153 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:29.154 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:29.155 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:29.155 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:29.155 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:29.155 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:29.155 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:29.155 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:29.155 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:29.156 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:29.156 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:29.156 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:29.156 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:29.156 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:33.157 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:33.158 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:33.159 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:33.159 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:33.159 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:33.159 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:33.159 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:33.159 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:33.159 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:33.159 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:33.159 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:33.160 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:33.160 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:33.160 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:37.161 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:37.163 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:37.163 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:37.163 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:37.163 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:37.163 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:37.163 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:37.163 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:37.163 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:37.164 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:37.164 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:37.164 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:37.164 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:37.164 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:41.165 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:41.167 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:41.167 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:41.167 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:41.167 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:41.167 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:41.167 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:41.168 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:41.168 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:41.168 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:41.168 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:41.168 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:41.168 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:41.168 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:45.171 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:45.173 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:45.174 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:45.174 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:45.174 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:45.175 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:45.175 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:45.176 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:45.176 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:45.176 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:45.177 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:45.177 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:45.177 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:45.177 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:49.178 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:49.179 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:49.180 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:49.180 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:49.180 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:49.181 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:49.181 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:49.182 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:49.182 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:49.182 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:49.182 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:49.183 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:49.183 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:49.183 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:53.181 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:53.182 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:53.182 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:53.183 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:53.183 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:53.183 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:53.183 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:53.183 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:53.183 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:53.183 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:53.184 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:53.184 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:53.184 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:53.184 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:57.187 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:57.188 7fd781158c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:57.188 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:57.189 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:57.189 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:57.189 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:57.189 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:57.190 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:57.190 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:57.190 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:57.190 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:57.190 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:57.191 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:57.191 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:01.193 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:01.195 7fd781158c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:01.195 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:01.195 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:01.195 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:01.195 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:01.196 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:01.196 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:01.196 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:01.197 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:01.197 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:01.197 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:01.197 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:01.197 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:05.198 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:05.200 7fd781158c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:05.200 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:05.201 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:05.201 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:05.202 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:05.202 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:05.203 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:05.203 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:05.203 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:05.204 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:05.204 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:05.204 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:05.204 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:09.211 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:09.220 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:09.221 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:09.221 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:09.221 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:09.222 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:09.222 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:09.223 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:09.223 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:09.224 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:09.224 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:09.224 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:09.225 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:09.225 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:13.224 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:13.226 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:13.226 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:13.226 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:13.226 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:13.227 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:13.227 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:13.227 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:13.227 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:13.228 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:13.228 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:13.228 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:13.228 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:13.228 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:17.227 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:17.230 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:17.230 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:17.230 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:17.231 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:17.231 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:17.231 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:17.232 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:17.232 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:17.233 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:17.233 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:17.233 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:17.234 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:17.234 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:21.235 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:21.238 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:21.238 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:21.238 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:21.239 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:21.239 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:21.239 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:21.240 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:21.241 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:21.241 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:21.241 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:21.242 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:21.242 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:21.242 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:21.696 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:01:22.292 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:22.292 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19376112 disk_space_available_1k = 61714372 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:01:22.292 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:22.590 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:01:23.913 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:01:25.238 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:25.242 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:25.242 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:25.242 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:25.243 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:25.243 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:25.243 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:25.244 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:25.245 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:25.245 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:25.245 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:25.246 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:25.246 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:25.246 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:26.962 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19377356, partition_space_available_1k=61713128, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1302300, cpu_share=0.06, mem_res=1293864, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45108, cpu_share=0.03, mem_res=42796, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=8726552, free=20670096, buffers=711260, cached=2557200, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.56, five_min_avg=2.78, fifteen_min_avg=2.05, cpu_share=3.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:01:26.963 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:26.964 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19377356 partition_space_available_1k = 61713128 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1302300 cpu_share = 0.06 mem_res = 1293864 >>contrail-database-nodemgr : << mem_virt = 45108 cpu_share = 0.03 mem_res = 42796 >> } system_mem_usage = << total = 32665108 used = 8726552 free = 20670096 buffers = 711260 cached = 2557200 node_type = database-node >> system_cpu_usage = << one_min_avg = 2.56 five_min_avg = 2.78 fifteen_min_avg = 2.05 cpu_share = 3.25 node_type = database-node >> >> 09/01/2025 02:01:26.964 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:29.244 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:29.245 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:29.246 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:29.246 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:29.246 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:29.246 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:29.246 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:29.246 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:29.247 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:29.247 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:29.247 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:29.247 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:29.247 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:29.247 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:33.249 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:33.251 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:33.252 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:33.252 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:33.252 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:33.252 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:33.252 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:33.253 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:33.253 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:33.254 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:33.254 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:33.254 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:33.254 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:33.254 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:37.256 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:37.258 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:37.259 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:37.259 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:37.260 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:37.260 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:37.260 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:37.262 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:37.262 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:37.263 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:37.263 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:37.263 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:37.264 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:37.264 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:41.260 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:41.262 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:41.262 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:41.263 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:41.263 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:41.263 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:41.263 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:41.264 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:41.264 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:41.265 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:41.265 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:41.265 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:41.266 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:41.266 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:45.265 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:45.268 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:45.268 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:45.269 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:45.269 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:45.270 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:45.270 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:45.271 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:45.271 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:45.272 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:45.272 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:45.272 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:45.273 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:45.273 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:49.270 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:49.272 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:49.273 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:49.273 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:49.274 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:49.275 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:49.275 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:49.276 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:49.276 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:49.277 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:49.277 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:49.277 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:49.278 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:49.278 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:53.275 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:53.277 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:53.278 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:53.278 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:53.278 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:53.278 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:53.278 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:53.278 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:53.278 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:53.278 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:53.279 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:53.279 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:53.279 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:53.279 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:57.280 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:57.282 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:57.283 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:57.283 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:57.283 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:57.284 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:57.284 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:57.285 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:57.285 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:57.286 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:57.286 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:57.286 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:57.286 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:57.287 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:01.285 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:01.287 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:01.287 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:01.287 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:01.287 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:01.287 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:01.287 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:01.287 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:01.288 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:01.288 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:01.288 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:01.288 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:01.288 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:01.288 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:05.289 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:05.291 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:05.291 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:05.291 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:05.291 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:05.291 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:05.291 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:05.292 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:05.292 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:05.292 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:05.292 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:05.292 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:05.292 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:05.292 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:09.294 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:09.296 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:09.296 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:09.297 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:09.297 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:09.297 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:09.297 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:09.298 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:09.298 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:09.298 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:09.298 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:09.298 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:09.298 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:09.298 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:13.299 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:13.301 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:13.301 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:13.302 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:13.302 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:13.303 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:13.303 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:13.304 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:13.304 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:13.305 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:13.305 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:13.305 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:13.306 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:13.306 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:17.306 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:17.308 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:17.308 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:17.309 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:17.309 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:17.309 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:17.310 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:17.310 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:17.311 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:17.311 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:17.311 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:17.312 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:17.312 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:17.312 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:21.315 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:21.327 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:21.327 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:21.328 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:21.328 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:21.328 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:21.328 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:21.329 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:21.330 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:21.330 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:21.330 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:21.330 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:21.331 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:21.331 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:21.768 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:02:22.405 7fd7811b0400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:22.406 7fd7811b0400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19367364 disk_space_available_1k = 61723120 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:02:22.406 7fd7811b0400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:22.840 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:02:23.946 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:02:25.327 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:25.329 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:25.329 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:25.329 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:25.329 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:25.330 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:25.330 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:25.330 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:25.330 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:25.330 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:25.330 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:25.331 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:25.331 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:25.331 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:27.068 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19367160, partition_space_available_1k=61723324, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1304652, cpu_share=0.04, mem_res=1293940, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45256, cpu_share=0.04, mem_res=42796, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9494488, free=19923616, buffers=714664, cached=2532340, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.66, five_min_avg=2.47, fifteen_min_avg=1.99, cpu_share=1.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:02:27.069 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:27.069 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19367160 partition_space_available_1k = 61723324 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1304652 cpu_share = 0.04 mem_res = 1293940 >>contrail-database-nodemgr : << mem_virt = 45256 cpu_share = 0.04 mem_res = 42796 >> } system_mem_usage = << total = 32665108 used = 9494488 free = 19923616 buffers = 714664 cached = 2532340 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.66 five_min_avg = 2.47 fifteen_min_avg = 1.99 cpu_share = 1.75 node_type = database-node >> >> 09/01/2025 02:02:27.069 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:29.330 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:29.332 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:29.332 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:29.333 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:29.333 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:29.333 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:29.334 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:29.335 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:29.335 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:29.336 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:29.336 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:29.336 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:29.337 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:29.337 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:33.333 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:33.334 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:33.334 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:33.334 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:33.334 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:33.335 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:33.335 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:33.335 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:33.335 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:33.335 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:33.335 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:33.335 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:33.335 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:33.336 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:37.338 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:37.340 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:37.340 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:37.341 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:37.341 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:37.341 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:37.342 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:37.343 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:37.343 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:37.344 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:37.344 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:37.344 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:37.345 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:37.345 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:40.716 7fd7811b0cc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:02:41.343 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:41.347 7fd7811cde00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:41.348 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:41.348 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:41.348 7fd7811cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:41.348 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:41.348 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:41.349 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:41.349 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:41.349 7fd7811cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:41.350 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:41.350 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:41.350 7fd7811cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:41.350 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:45.350 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:45.352 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:45.352 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:45.353 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:45.353 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:45.353 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:45.354 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:45.354 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:45.355 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:45.355 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:45.355 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:45.356 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:45.356 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:45.356 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:49.357 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:49.359 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:49.360 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:49.360 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:49.361 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:49.361 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:49.361 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:49.362 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:49.363 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:49.363 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:49.363 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:49.363 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:49.364 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:49.364 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:53.363 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:53.364 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:53.365 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:53.365 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:53.365 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:53.366 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:53.366 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:53.367 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:53.367 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:53.367 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:53.367 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:53.368 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:53.368 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:53.368 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:53.600 7fd7811cd7c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:02:57.365 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:57.367 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:57.367 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:57.367 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:57.367 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:57.367 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:57.367 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:57.367 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:57.368 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:57.368 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:57.368 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:57.368 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:57.368 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:57.368 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:01.372 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:01.374 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:01.375 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:01.375 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:01.376 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:01.376 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:01.376 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:01.378 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:01.378 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:01.379 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:01.379 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:01.380 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:01.380 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:01.380 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:05.379 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:05.381 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:05.382 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:05.382 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:05.383 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:05.383 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:05.384 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:05.385 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:05.385 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:05.386 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:05.386 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:05.387 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:05.387 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:05.388 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:06.621 7fd7811b07c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:03:09.384 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:09.386 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:09.386 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:09.387 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:09.387 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:09.387 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:09.388 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:09.388 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:09.389 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:09.389 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:09.390 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:09.390 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:09.390 7fd7811b07c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:09.390 7fd7811b07c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:13.388 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:13.391 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:13.391 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:13.391 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:13.392 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:13.392 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:13.392 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:13.393 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:13.393 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:13.394 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:13.394 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:13.394 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:13.394 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:13.395 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:17.394 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:17.396 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:17.397 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:17.397 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:17.398 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:17.398 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:17.398 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:17.400 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:17.400 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:17.400 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:17.401 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:17.401 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:17.402 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:17.402 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:19.615 7fd7811b0860 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:03:21.399 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:21.403 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:21.403 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:21.403 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:21.403 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:21.403 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:21.403 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:21.404 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:21.404 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:21.404 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:21.404 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:21.404 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:21.404 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:21.405 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:21.684 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:03:22.409 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:22.410 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19549384 disk_space_available_1k = 61541100 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:03:22.411 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:22.804 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:03:24.081 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:03:25.404 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:25.407 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:25.408 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:25.408 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:25.408 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:25.409 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:25.409 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:25.410 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:25.410 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:25.410 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:25.411 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:25.411 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:25.411 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:25.412 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:27.161 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19549520, partition_space_available_1k=61540964, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1305104, cpu_share=0.05, mem_res=1294016, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45496, cpu_share=0.04, mem_res=42796, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9329184, free=19838400, buffers=720172, cached=2777352, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.21, five_min_avg=2.19, fifteen_min_avg=1.92, cpu_share=2.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:03:27.162 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:27.162 7fd781158e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19549520 partition_space_available_1k = 61540964 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1305104 cpu_share = 0.05 mem_res = 1294016 >>contrail-database-nodemgr : << mem_virt = 45496 cpu_share = 0.04 mem_res = 42796 >> } system_mem_usage = << total = 32665108 used = 9329184 free = 19838400 buffers = 720172 cached = 2777352 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.21 five_min_avg = 2.19 fifteen_min_avg = 1.92 cpu_share = 2.0 node_type = database-node >> >> 09/01/2025 02:03:27.163 7fd781158e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:29.410 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:29.413 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:29.413 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:29.413 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:29.414 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:29.414 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:29.414 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:29.415 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:29.415 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:29.416 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:29.417 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:29.417 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:29.418 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:29.418 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:32.479 7fd7811cd220 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:03:33.414 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:33.416 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:33.416 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:33.417 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:33.417 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:33.417 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:33.418 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:33.418 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:33.419 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:33.419 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:33.419 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:33.420 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:33.420 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:33.420 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:37.421 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:37.423 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:37.423 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:37.424 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:37.424 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:37.424 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:37.424 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:37.424 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:37.425 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:37.425 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:37.425 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:37.425 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:37.425 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:37.425 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:41.425 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:41.428 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:41.428 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:41.428 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:41.429 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:41.429 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:41.429 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:41.430 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:41.430 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:41.431 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:41.431 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:41.431 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:41.431 7fd7811cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:41.432 7fd7811cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:45.429 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:45.431 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:45.431 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:45.431 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:45.431 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:45.431 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:45.431 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:45.431 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:45.431 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:45.432 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:45.432 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:45.432 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:45.432 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:45.432 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:45.812 7fd7811b09a0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:03:49.433 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:49.434 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:49.435 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:49.435 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:49.435 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:49.435 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:49.435 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:49.435 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:49.435 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:49.436 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:49.436 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:49.436 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:49.436 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:49.436 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:53.439 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:53.441 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:53.442 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:53.442 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:53.442 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:53.442 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:53.443 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:53.443 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:53.444 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:53.444 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:53.444 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:53.444 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:53.445 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:53.445 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:57.446 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:57.449 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:57.449 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:57.450 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:57.450 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:57.450 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:57.450 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:57.451 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:57.452 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:57.452 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:57.452 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:57.452 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:57.453 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:57.453 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:58.595 7fd781d37d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:04:01.451 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:01.454 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:01.454 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:01.455 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:01.455 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:01.456 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:01.456 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:01.457 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:01.457 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:01.458 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:01.458 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:01.459 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:01.459 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:01.459 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:05.458 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:05.460 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:05.461 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:05.461 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:05.462 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:05.462 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:05.463 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:05.464 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:05.464 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:05.465 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:05.465 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:05.466 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:05.466 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:05.466 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:09.462 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:09.464 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:09.464 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:09.465 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:09.465 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:09.465 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:09.466 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:09.466 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:09.467 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:09.467 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:09.467 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:09.467 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:09.468 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:09.468 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:11.593 7fd781d37d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:04:13.465 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:13.468 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:13.468 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:13.468 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:13.469 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:13.469 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:13.470 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:13.470 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:13.471 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:13.471 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:13.471 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:13.472 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:13.472 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:13.472 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:17.469 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:17.472 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:17.472 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:17.472 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:17.473 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:17.473 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:17.473 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:17.474 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:17.475 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:17.475 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:17.475 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:17.475 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:17.476 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:17.476 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:21.472 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:21.477 7fd781d1e360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:21.478 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:21.478 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:21.478 7fd781d1e360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:21.479 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:21.479 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:21.479 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:21.480 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:21.480 7fd781d1e360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:21.480 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:21.480 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:21.480 7fd781d1e360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:21.481 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:21.762 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:04:22.402 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:22.402 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19552772 disk_space_available_1k = 61537712 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:04:22.402 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:22.916 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:04:24.400 7fd781d37d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:04:24.415 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:04:25.478 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:25.481 7fd781158c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:25.481 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:25.482 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:25.482 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:25.482 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:25.482 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:25.483 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:25.483 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:25.483 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:25.483 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:25.484 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:25.484 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:25.484 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:27.461 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19557460, partition_space_available_1k=61533024, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1303880, cpu_share=0.07, mem_res=1292476, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45480, cpu_share=0.04, mem_res=42796, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9344048, free=19819168, buffers=720236, cached=2781656, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.75, five_min_avg=2.13, fifteen_min_avg=1.91, cpu_share=1.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:04:27.461 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:27.462 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19557460 partition_space_available_1k = 61533024 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1303880 cpu_share = 0.07 mem_res = 1292476 >>contrail-database-nodemgr : << mem_virt = 45480 cpu_share = 0.04 mem_res = 42796 >> } system_mem_usage = << total = 32665108 used = 9344048 free = 19819168 buffers = 720236 cached = 2781656 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.75 five_min_avg = 2.13 fifteen_min_avg = 1.91 cpu_share = 1.75 node_type = database-node >> >> 09/01/2025 02:04:27.462 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:29.483 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:29.485 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:29.486 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:29.486 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:29.486 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:29.487 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:29.487 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:29.489 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:29.489 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:29.489 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:29.490 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:29.490 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:29.490 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:29.491 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:33.490 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:33.492 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:33.492 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:33.492 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:33.493 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:33.493 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:33.493 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:33.494 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:33.494 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:33.495 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:33.495 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:33.495 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:33.496 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:33.496 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:37.250 7fd781158f40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:04:37.492 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:37.493 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:37.493 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:37.493 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:37.493 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:37.494 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:37.494 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:37.494 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:37.494 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:37.494 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:37.494 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:37.494 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:37.494 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:37.494 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:41.497 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:41.499 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:41.499 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:41.499 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:41.499 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:41.499 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:41.499 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:41.499 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:41.500 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:41.500 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:41.500 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:41.500 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:41.500 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:41.500 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:45.501 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:45.503 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:45.504 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:45.504 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:45.504 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:45.505 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:45.505 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:45.506 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:45.506 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:45.507 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:45.507 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:45.507 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:45.508 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:45.508 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:49.508 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:49.510 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:49.511 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:49.511 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:49.511 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:49.512 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:49.512 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:49.513 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:49.513 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:49.513 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:49.513 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:49.514 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:49.514 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:49.514 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:50.482 7fd7811cd7c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:04:53.514 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:53.515 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:53.516 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:53.516 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:53.516 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:53.517 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:53.517 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:53.518 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:53.518 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:53.518 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:53.518 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:53.519 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:53.519 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:53.519 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:57.518 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:57.520 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:57.521 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:57.521 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:57.522 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:57.522 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:57.522 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:57.523 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:57.523 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:57.523 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:57.524 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:57.524 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:57.524 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:57.524 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:01.522 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:01.523 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:01.524 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:01.524 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:01.524 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:01.524 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:01.525 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:01.525 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:01.526 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:01.526 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:01.526 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:01.526 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:01.526 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:01.527 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:03.305 7fd7811b0cc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:05.527 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:05.528 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:05.528 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:05.528 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:05.529 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:05.529 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:05.529 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:05.529 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:05.529 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:05.530 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:05.530 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:05.530 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:05.530 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:05.530 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:09.533 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:09.535 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:09.536 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:09.536 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:09.536 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:09.537 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:09.537 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:09.538 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:09.538 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:09.538 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:09.538 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:09.539 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:09.539 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:09.539 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:13.538 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:13.540 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:13.540 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:13.540 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:13.541 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:13.541 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:13.541 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:13.542 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:13.542 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:13.543 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:13.543 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:13.543 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:13.543 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:13.544 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:16.152 7fd781d37d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:17.542 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:17.544 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:17.545 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:17.545 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:17.546 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:17.546 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:17.546 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:17.547 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:17.547 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:17.548 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:17.548 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:17.548 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:17.548 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:17.549 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:21.545 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:21.547 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:21.548 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:21.548 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:21.549 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:21.549 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:21.549 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:21.550 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:21.550 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:21.551 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:21.551 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:21.551 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:21.551 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:21.552 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:21.749 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:05:22.526 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:22.526 7fd781d37d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19556064 disk_space_available_1k = 61534420 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:05:22.527 7fd781d37d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:22.871 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:05:24.079 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:05:25.548 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:25.550 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:25.550 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:25.550 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:25.551 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:25.551 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:25.551 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:25.551 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:25.551 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:25.552 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:25.552 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:25.552 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:25.552 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:25.552 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:27.124 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19556188, partition_space_available_1k=61534296, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1303992, cpu_share=0.05, mem_res=1292552, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45564, cpu_share=0.04, mem_res=42796, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9342808, free=19816924, buffers=720264, cached=2785112, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.25, five_min_avg=1.92, fifteen_min_avg=1.86, cpu_share=1.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:05:27.124 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:27.125 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19556188 partition_space_available_1k = 61534296 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1303992 cpu_share = 0.05 mem_res = 1292552 >>contrail-database-nodemgr : << mem_virt = 45564 cpu_share = 0.04 mem_res = 42796 >> } system_mem_usage = << total = 32665108 used = 9342808 free = 19816924 buffers = 720264 cached = 2785112 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.25 five_min_avg = 1.92 fifteen_min_avg = 1.86 cpu_share = 1.75 node_type = database-node >> >> 09/01/2025 02:05:27.125 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:29.088 7fd7811cd900 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:29.555 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:29.562 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:29.562 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:29.562 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:29.563 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:29.563 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:29.563 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:29.564 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:29.565 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:29.565 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:29.566 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:29.566 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:29.566 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:29.567 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:33.566 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:33.568 7fd781158e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:33.569 7fd781158e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:33.569 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:33.569 7fd781158e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:33.570 7fd781158e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:33.570 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:33.571 7fd781158e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:33.571 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:33.572 7fd781158e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:33.572 7fd781158e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:33.572 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:33.573 7fd781158e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:33.573 7fd781158e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:37.571 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:37.573 7fd781158e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:37.574 7fd781158e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:37.574 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:37.575 7fd781158e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:37.575 7fd781158e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:37.575 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:37.576 7fd781158e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:37.576 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:37.577 7fd781158e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:37.577 7fd781158e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:37.577 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:37.577 7fd781158e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:37.578 7fd781158e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:41.578 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:41.581 7fd781158e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:41.582 7fd781158e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:41.582 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:41.583 7fd781158e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:41.583 7fd781158e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:41.584 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:41.584 7fd781158e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:41.585 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:41.585 7fd781158e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:41.585 7fd781158e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:41.586 7fd781158e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:41.586 7fd781158e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:41.586 7fd781158e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:42.011 7fd781158e00 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:45.583 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:45.585 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:45.586 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:45.586 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:45.586 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:45.586 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:45.587 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:45.588 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:45.588 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:45.588 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:45.589 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:45.589 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:45.589 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:45.589 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:49.590 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:49.593 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:49.593 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:49.594 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:49.594 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:49.594 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:49.595 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:49.596 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:49.596 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:49.596 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:49.597 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:49.597 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:49.597 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:49.598 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:53.598 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:53.599 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:53.599 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:53.599 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:53.599 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:53.599 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:53.599 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:53.600 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:53.600 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:53.600 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:53.600 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:53.600 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:53.601 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:53.601 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:55.197 7fd7811cd7c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:57.602 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:57.604 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:57.605 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:57.605 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:57.606 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:57.606 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:57.606 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:57.607 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:57.607 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:57.608 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:57.608 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:57.608 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:57.609 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:57.609 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:01.606 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:01.608 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:01.609 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:01.609 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:01.610 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:01.610 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:01.610 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:01.611 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:01.611 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:01.612 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:01.612 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:01.612 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:01.612 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:01.613 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:05.610 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:05.612 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:05.613 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:05.613 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:05.614 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:05.614 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:05.615 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:05.616 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:05.616 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:05.616 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:05.617 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:05.617 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:05.618 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:05.618 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:08.178 7fd7811b0360 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:06:09.614 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:09.617 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:09.617 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:09.617 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:09.618 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:09.618 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:09.618 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:09.619 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:09.620 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:09.620 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:09.620 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:09.620 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:09.621 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:09.621 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:13.622 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:13.624 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:13.624 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:13.625 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:13.625 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:13.626 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:13.626 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:13.627 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:13.627 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:13.627 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:13.627 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:13.628 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:13.628 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:13.628 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:17.627 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:17.628 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:17.628 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:17.628 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:17.628 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:17.628 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:17.628 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:17.629 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:17.629 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:17.629 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:17.629 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:17.629 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:17.629 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:17.629 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:21.185 7fd7811cdae0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:06:21.681 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:06:21.683 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:21.683 7fd7811b0220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:21.684 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:21.684 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:21.684 7fd7811b0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:21.684 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:21.684 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:21.684 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:21.684 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:21.684 7fd7811b0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:21.684 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:21.684 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:21.684 7fd7811b0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:21.685 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:22.472 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:22.473 7fd7811b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19559820 disk_space_available_1k = 61530664 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:06:22.473 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:22.756 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:06:23.982 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:06:25.685 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:25.689 7fd7811b0220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:25.690 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:25.690 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:25.690 7fd7811b0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:25.691 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:25.691 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:25.692 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:25.693 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:25.693 7fd7811b0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:25.694 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:25.694 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:25.694 7fd7811b0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:25.695 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:27.032 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19559940, partition_space_available_1k=61530544, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1304096, cpu_share=0.05, mem_res=1292628, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45656, cpu_share=0.04, mem_res=42796, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9339416, free=19816100, buffers=720344, cached=2789248, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.59, five_min_avg=1.91, fifteen_min_avg=1.86, cpu_share=1.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:06:27.033 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:27.033 7fd781d1e360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19559940 partition_space_available_1k = 61530544 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1304096 cpu_share = 0.05 mem_res = 1292628 >>contrail-database-nodemgr : << mem_virt = 45656 cpu_share = 0.04 mem_res = 42796 >> } system_mem_usage = << total = 32665108 used = 9339416 free = 19816100 buffers = 720344 cached = 2789248 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.59 five_min_avg = 1.91 fifteen_min_avg = 1.86 cpu_share = 1.75 node_type = database-node >> >> 09/01/2025 02:06:27.034 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:29.690 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:29.692 7fd781d1e360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:29.693 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:29.693 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:29.694 7fd781d1e360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:29.694 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:29.694 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:29.696 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:29.696 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:29.696 7fd781d1e360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:29.697 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:29.697 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:29.698 7fd781d1e360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:29.698 7fd781d1e360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:33.694 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:33.696 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:33.697 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:33.697 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:33.698 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:33.698 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:33.699 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:33.700 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:33.700 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:33.701 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:33.701 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:33.701 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:33.702 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:33.702 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:33.948 7fd7811b0040 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:06:37.700 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:37.702 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:37.703 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:37.703 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:37.704 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:37.704 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:37.704 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:37.705 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:37.706 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:37.706 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:37.707 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:37.707 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:37.707 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:37.708 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:41.705 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:41.707 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:41.707 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:41.707 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:41.707 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:41.707 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:41.707 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:41.708 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:41.708 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:41.708 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:41.708 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:41.708 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:41.708 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:41.708 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:45.710 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:45.712 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:45.712 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:45.712 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:45.713 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:45.713 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:45.713 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:45.714 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:45.714 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:45.715 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:45.715 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:45.715 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:45.715 7fd7811b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:45.716 7fd7811b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:46.840 7fd7811cd900 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:06:49.714 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:49.716 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:49.716 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:49.717 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:49.717 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:49.717 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:49.718 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:49.719 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:49.719 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:49.719 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:49.719 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:49.720 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:49.720 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:49.720 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:53.722 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:53.723 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:53.724 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:53.724 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:53.724 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:53.724 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:53.725 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:53.725 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:53.726 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:53.726 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:53.726 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:53.726 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:53.727 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:53.727 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:57.726 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:57.728 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:57.728 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:57.728 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:57.729 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:57.729 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:57.730 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:57.731 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:57.731 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:57.732 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:57.732 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:57.733 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:57.733 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:57.733 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:00.033 7fd7811cd900 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:07:01.728 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:01.729 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:01.730 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:01.730 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:01.730 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:01.730 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:01.730 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:01.730 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:01.730 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:01.731 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:01.731 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:01.731 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:01.731 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:01.731 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:05.733 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:05.735 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:05.735 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:05.735 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:05.736 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:05.736 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:05.736 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:05.737 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:05.737 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:05.737 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:05.737 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:05.737 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:05.738 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:05.738 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:09.740 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:09.743 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:09.743 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:09.744 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:09.744 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:09.744 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:09.745 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:09.745 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:09.746 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:09.746 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:09.747 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:09.747 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:09.747 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:09.747 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:13.039 7fd7811b0cc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:07:13.744 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:13.746 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:13.747 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:13.747 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:13.748 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:13.748 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:13.748 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:13.749 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:13.750 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:13.750 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:13.750 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:13.751 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:13.751 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:13.751 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:17.748 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:17.750 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:17.751 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:17.751 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:17.751 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:17.752 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:17.752 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:17.753 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:17.753 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:17.753 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:17.754 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:17.754 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:17.754 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:17.754 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:21.704 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:07:21.753 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:21.760 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:21.760 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:21.761 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:21.761 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:21.761 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:21.762 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:21.762 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:21.763 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:21.763 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:21.763 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:21.764 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:21.764 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:21.764 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:22.543 7fd7811cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:22.544 7fd7811cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19563536 disk_space_available_1k = 61526948 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:07:22.544 7fd7811cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:22.832 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:07:24.027 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:07:25.761 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:25.763 7fd7811b00e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:25.763 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:25.763 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:25.764 7fd7811b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:25.764 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:25.764 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:25.764 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:25.764 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:25.764 7fd7811b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:25.765 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:25.765 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:25.765 7fd7811b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:25.765 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:26.147 7fd7811b00e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:07:27.081 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19563664, partition_space_available_1k=61526820, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1304196, cpu_share=0.05, mem_res=1292692, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45748, cpu_share=0.04, mem_res=42796, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9374436, free=19776916, buffers=720588, cached=2793168, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.9, five_min_avg=1.91, fifteen_min_avg=1.86, cpu_share=1.62, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:07:27.082 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:27.082 7fd7811b02c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19563664 partition_space_available_1k = 61526820 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1304196 cpu_share = 0.05 mem_res = 1292692 >>contrail-database-nodemgr : << mem_virt = 45748 cpu_share = 0.04 mem_res = 42796 >> } system_mem_usage = << total = 32665108 used = 9374436 free = 19776916 buffers = 720588 cached = 2793168 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.9 five_min_avg = 1.91 fifteen_min_avg = 1.86 cpu_share = 1.62 node_type = database-node >> >> 09/01/2025 02:07:27.083 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:29.765 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:29.768 7fd7811b02c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:29.768 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:29.769 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:29.769 7fd7811b02c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:29.770 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:29.770 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:29.771 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:29.771 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:29.772 7fd7811b02c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:29.772 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:29.772 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:29.773 7fd7811b02c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:29.773 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:33.770 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:33.772 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:33.772 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:33.773 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:33.773 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:33.773 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:33.773 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:33.774 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:33.775 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:33.775 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:33.776 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:33.776 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:33.776 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:33.776 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:37.777 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:37.780 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:37.780 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:37.781 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:37.781 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:37.781 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:37.782 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:37.783 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:37.783 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:37.783 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:37.783 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:37.784 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:37.784 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:37.784 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:38.897 7fd7811b0180 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:07:41.782 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:41.785 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:41.786 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:41.786 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:41.786 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:41.787 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:41.787 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:41.788 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:41.788 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:41.788 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:41.789 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:41.789 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:41.789 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:41.790 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:45.789 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:45.792 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:45.792 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:45.792 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:45.793 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:45.793 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:45.793 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:45.794 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:45.794 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:45.795 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:45.795 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:45.795 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:45.796 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:45.796 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:49.794 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:49.796 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:49.796 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:49.796 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:49.797 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:49.797 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:49.797 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:49.798 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:49.799 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:49.799 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:49.799 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:49.799 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:49.800 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:49.800 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:51.648 7fd7811b0cc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:07:53.798 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:53.800 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:53.801 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:53.801 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:53.802 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:53.802 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:53.802 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:53.803 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:53.803 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:53.804 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:53.804 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:53.804 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:53.805 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:53.805 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:57.802 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:57.804 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:57.804 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:57.805 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:57.805 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:57.805 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:57.806 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:57.807 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:57.807 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:57.807 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:57.808 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:57.808 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:57.808 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:57.809 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:01.806 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:01.807 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:01.807 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:01.807 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:01.808 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:01.808 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:01.808 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:01.808 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:01.808 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:01.808 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:01.808 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:01.808 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:01.809 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:01.809 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:04.667 7fd7811cdcc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:08:05.809 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:05.811 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:05.812 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:05.812 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:05.812 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:05.813 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:05.813 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:05.814 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:05.814 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:05.814 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:05.815 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:05.815 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:05.815 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:05.815 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:09.816 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:09.817 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:09.817 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:09.817 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:09.818 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:09.818 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:09.818 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:09.818 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:09.819 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:09.819 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:09.819 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:09.819 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:09.819 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:09.819 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:13.820 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:13.823 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:13.823 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:13.824 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:13.824 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:13.824 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:13.825 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:13.826 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:13.826 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:13.826 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:13.827 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:13.827 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:13.827 7fd7811cdcc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:13.828 7fd7811cdcc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:17.778 7fd7811b02c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:08:17.829 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:17.837 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:17.838 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:17.838 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:17.839 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:17.839 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:17.839 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:17.840 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:17.841 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:17.841 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:17.841 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:17.841 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:17.842 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:17.842 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:21.639 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:08:21.838 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:21.839 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:21.839 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:21.840 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:21.840 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:21.840 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:21.840 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:21.840 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:21.840 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:21.841 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:21.841 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:21.841 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:21.841 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:21.841 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:22.168 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:22.168 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19566780 disk_space_available_1k = 61523704 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:08:22.168 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:22.454 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:08:23.840 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:08:25.841 7fd7811cdf40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:25.845 7fd7811cdf40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:25.846 7fd7811cdf40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:25.846 7fd7811cdf40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:25.846 7fd7811cdf40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:25.847 7fd7811cdf40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:25.847 7fd7811cdf40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:25.848 7fd7811cdf40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:25.848 7fd7811cdf40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:25.848 7fd7811cdf40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:25.849 7fd7811cdf40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:25.849 7fd7811cdf40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:25.849 7fd7811cdf40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:25.850 7fd7811cdf40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:26.905 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19566908, partition_space_available_1k=61523576, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1304300, cpu_share=0.05, mem_res=1292768, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45840, cpu_share=0.03, mem_res=42796, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9380032, free=19765664, buffers=720648, cached=2798764, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.38, five_min_avg=1.76, fifteen_min_avg=1.81, cpu_share=1.62, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:08:26.906 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:26.906 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19566908 partition_space_available_1k = 61523576 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1304300 cpu_share = 0.05 mem_res = 1292768 >>contrail-database-nodemgr : << mem_virt = 45840 cpu_share = 0.03 mem_res = 42796 >> } system_mem_usage = << total = 32665108 used = 9380032 free = 19765664 buffers = 720648 cached = 2798764 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.38 five_min_avg = 1.76 fifteen_min_avg = 1.81 cpu_share = 1.62 node_type = database-node >> >> 09/01/2025 02:08:26.906 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:29.848 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:29.851 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:29.852 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:29.852 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:29.853 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:29.853 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:29.854 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:29.855 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:29.855 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:29.856 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:29.856 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:29.857 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:29.857 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:29.857 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:30.540 7fd781158ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:08:33.852 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:33.855 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:33.856 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:33.856 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:33.856 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:33.857 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:33.857 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:33.858 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:33.858 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:33.859 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:33.859 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:33.859 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:33.860 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:33.860 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:37.859 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:37.861 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:37.861 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:37.862 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:37.862 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:37.862 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:37.863 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:37.863 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:37.864 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:37.864 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:37.864 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:37.865 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:37.865 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:37.865 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:41.865 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:41.866 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:41.867 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:41.867 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:41.867 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:41.867 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:41.867 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:41.867 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:41.867 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:41.868 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:41.868 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:41.868 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:41.868 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:41.868 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:43.215 7fd7811cd900 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:08:45.870 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:45.871 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:45.871 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:45.871 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:45.871 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:45.872 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:45.872 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:45.872 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:45.872 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:45.872 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:45.872 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:45.873 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:45.873 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:45.873 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:49.873 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:49.875 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:49.875 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:49.876 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:49.876 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:49.876 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:49.877 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:49.877 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:49.878 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:49.878 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:49.878 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:49.879 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:49.879 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:49.879 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:53.877 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:53.879 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:53.880 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:53.880 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:53.880 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:53.880 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:53.881 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:53.881 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:53.881 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:53.882 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:53.882 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:53.882 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:53.882 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:53.883 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:55.866 7fd7811cd7c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:08:57.882 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:57.884 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:57.884 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:57.885 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:57.885 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:57.886 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:57.886 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:57.887 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:57.887 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:57.888 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:57.888 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:57.888 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:57.889 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:57.889 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:01.886 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:01.888 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:01.889 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:01.889 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:01.890 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:01.890 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:01.890 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:01.891 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:01.892 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:01.892 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:01.893 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:01.893 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:01.893 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:01.894 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:05.891 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:05.892 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:05.892 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:05.892 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:05.892 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:05.892 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:05.892 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:05.893 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:05.893 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:05.893 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:05.893 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:05.893 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:05.893 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:05.893 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:08.994 7fd781158f40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:09:09.893 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:09.895 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:09.896 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:09.896 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:09.896 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:09.897 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:09.897 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:09.898 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:09.898 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:09.899 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:09.899 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:09.899 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:09.899 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:09.900 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:13.897 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:13.900 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:13.900 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:13.901 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:13.901 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:13.901 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:13.901 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:13.902 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:13.903 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:13.903 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:13.903 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:13.904 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:13.904 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:13.904 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:17.901 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:17.904 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:17.904 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:17.904 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:17.905 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:17.905 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:17.906 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:17.906 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:17.907 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:17.907 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:17.907 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:17.908 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:17.908 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:17.908 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:21.658 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:09:21.807 7fd781158b80 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:09:21.904 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:21.905 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:21.906 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:21.906 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:21.906 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:21.906 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:21.906 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:21.907 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:21.907 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:21.907 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:21.907 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:21.907 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:21.908 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:21.908 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:22.484 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:22.485 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19574772 disk_space_available_1k = 61515712 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:09:22.485 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:22.803 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:09:24.004 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:09:25.907 7fd7811cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:25.911 7fd7811cdb80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:25.911 7fd7811cdb80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:25.911 7fd7811cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:25.912 7fd7811cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:25.912 7fd7811cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:25.912 7fd7811cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:25.913 7fd7811cdb80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:25.913 7fd7811cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:25.914 7fd7811cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:25.914 7fd7811cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:25.914 7fd7811cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:25.915 7fd7811cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:25.915 7fd7811cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:27.060 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19570360, partition_space_available_1k=61520124, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1304728, cpu_share=0.06, mem_res=1292984, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45936, cpu_share=0.03, mem_res=42796, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9371120, free=19773084, buffers=720728, cached=2800176, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.26, five_min_avg=1.67, fifteen_min_avg=1.78, cpu_share=1.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:09:27.061 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:27.062 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19570360 partition_space_available_1k = 61520124 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1304728 cpu_share = 0.06 mem_res = 1292984 >>contrail-database-nodemgr : << mem_virt = 45936 cpu_share = 0.03 mem_res = 42796 >> } system_mem_usage = << total = 32665108 used = 9371120 free = 19773084 buffers = 720728 cached = 2800176 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.26 five_min_avg = 1.67 fifteen_min_avg = 1.78 cpu_share = 1.75 node_type = database-node >> >> 09/01/2025 02:09:27.062 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:29.914 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:29.916 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:29.917 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:29.917 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:29.917 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:29.918 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:29.918 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:29.919 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:29.919 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:29.919 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:29.920 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:29.920 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:29.920 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:29.921 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:33.921 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:33.925 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:33.925 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:33.925 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:33.926 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:33.926 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:33.926 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:33.927 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:33.928 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:33.928 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:33.928 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:33.928 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:33.929 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:33.929 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:34.741 7fd7811b0d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:09:37.927 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:37.928 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:37.928 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:37.928 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:37.928 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:37.928 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:37.928 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:37.929 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:37.929 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:37.929 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:37.929 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:37.929 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:37.929 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:37.929 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:41.929 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:41.931 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:41.931 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:41.931 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:41.931 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:41.931 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:41.931 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:41.932 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:41.932 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:41.932 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:41.932 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:41.932 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:41.932 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:41.932 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:45.934 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:45.936 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:45.937 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:45.938 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:45.938 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:45.938 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:45.939 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:45.940 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:45.940 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:45.941 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:45.941 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:45.942 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:45.942 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:45.943 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:47.728 7fd7811b0d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:09:49.937 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:49.940 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:49.940 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:49.940 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:49.941 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:49.941 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:49.942 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:49.943 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:49.943 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:49.944 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:49.944 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:49.944 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:49.945 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:49.945 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:53.944 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:53.946 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:53.946 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:53.947 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:53.947 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:53.947 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:53.948 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:53.948 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:53.949 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:53.949 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:53.949 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:53.949 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:53.950 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:53.950 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:57.950 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:57.952 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:57.953 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:57.953 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:57.953 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:57.954 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:57.954 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:57.955 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:57.955 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:57.955 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:57.956 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:57.956 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:57.956 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:57.957 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:00.318 7fd7811cdae0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:10:01.954 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:01.956 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:01.956 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:01.956 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:01.957 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:01.957 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:01.957 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:01.958 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:01.958 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:01.958 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:01.959 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:01.959 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:01.959 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:01.959 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:05.957 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:05.959 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:05.959 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:05.959 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:05.959 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:05.959 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:05.959 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:05.960 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:05.960 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:05.960 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:05.960 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:05.960 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:05.960 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:05.961 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:09.963 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:09.965 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:09.966 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:09.966 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:09.966 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:09.967 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:09.967 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:09.968 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:09.968 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:09.968 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:09.969 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:09.969 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:09.969 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:09.970 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:13.295 7fd7811b0720 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:10:13.967 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:13.969 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:13.969 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:13.970 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:13.970 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:13.970 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:13.971 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:13.971 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:13.972 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:13.972 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:13.972 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:13.972 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:13.973 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:13.973 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:17.973 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:17.975 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:17.976 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:17.976 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:17.976 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:17.977 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:17.977 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:17.978 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:17.978 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:17.979 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:17.979 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:17.979 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:17.980 7fd7811b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:17.980 7fd7811b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:21.726 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:10:21.976 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:21.978 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:21.979 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:21.979 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:21.979 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:21.980 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:21.980 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:21.981 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:21.981 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:21.982 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:21.982 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:21.982 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:21.982 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:21.983 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:22.349 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:22.350 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19573500 disk_space_available_1k = 61516984 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:10:22.351 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:22.776 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:10:23.993 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:10:25.980 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:25.985 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:25.985 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:25.986 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:25.986 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:25.987 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:25.987 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:25.988 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:25.988 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:25.989 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:25.989 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:25.989 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:25.990 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:25.990 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:26.058 7fd7811b0d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:10:27.047 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19573620, partition_space_available_1k=61516864, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1304836, cpu_share=0.05, mem_res=1293060, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46028, cpu_share=0.04, mem_res=42796, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9423512, free=19711672, buffers=720812, cached=2809112, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.26, five_min_avg=1.58, fifteen_min_avg=1.74, cpu_share=1.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:10:27.048 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:27.048 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19573620 partition_space_available_1k = 61516864 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1304836 cpu_share = 0.05 mem_res = 1293060 >>contrail-database-nodemgr : << mem_virt = 46028 cpu_share = 0.04 mem_res = 42796 >> } system_mem_usage = << total = 32665108 used = 9423512 free = 19711672 buffers = 720812 cached = 2809112 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.26 five_min_avg = 1.58 fifteen_min_avg = 1.74 cpu_share = 1.75 node_type = database-node >> >> 09/01/2025 02:10:27.049 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:29.988 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:29.992 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:29.992 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:29.993 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:29.993 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:29.994 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:29.994 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:29.995 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:29.996 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:29.996 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:29.996 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:29.997 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:29.997 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:29.998 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:33.997 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:33.999 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:33.999 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:34.000 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:34.000 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:34.001 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:34.001 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:34.003 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:34.003 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:34.004 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:34.004 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:34.005 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:34.005 7fd7811cd900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:34.005 7fd7811cd900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:38.002 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:38.004 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:38.005 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:38.005 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:38.006 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:38.006 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:38.006 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:38.007 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:38.008 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:38.008 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:38.008 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:38.009 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:38.009 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:38.009 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:39.085 7fd7811b09a0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:10:42.006 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:42.009 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:42.010 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:42.010 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:42.011 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:42.011 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:42.011 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:42.013 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:42.013 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:42.013 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:42.014 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:42.014 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:42.014 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:42.015 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:46.014 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:46.016 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:46.017 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:46.017 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:46.018 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:46.018 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:46.018 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:46.019 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:46.019 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:46.020 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:46.020 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:46.020 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:46.021 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:46.021 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:50.017 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:50.019 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:50.019 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:50.019 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:50.019 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:50.019 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:50.019 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:50.020 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:50.020 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:50.020 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:50.020 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:50.020 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:50.020 7fd7811b09a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:50.020 7fd7811b09a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:52.029 7fd7811b0900 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:10:54.020 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:54.023 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:54.023 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:54.024 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:54.024 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:54.024 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:54.024 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:54.025 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:54.025 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:54.025 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:54.025 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:54.026 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:54.026 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:54.026 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:58.024 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:58.025 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:58.025 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:58.025 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:58.026 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:58.026 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:58.026 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:58.026 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:58.026 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:58.026 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:58.026 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:58.026 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:58.027 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:58.027 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:02.030 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:02.032 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:02.032 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:02.032 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:02.033 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:02.033 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:02.033 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:02.034 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:02.034 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:02.035 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:02.035 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:02.035 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:02.036 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:02.036 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:04.828 7fd7811b0900 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:11:06.034 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:06.035 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:06.035 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:06.035 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:06.036 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:06.036 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:06.036 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:06.036 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:06.036 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:06.036 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:06.036 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:06.036 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:06.037 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:06.037 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:10.039 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:10.042 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:10.043 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:10.043 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:10.043 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:10.044 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:10.044 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:10.045 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:10.045 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:10.046 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:10.046 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:10.046 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:10.047 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:10.047 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:14.044 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:14.047 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:14.047 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:14.048 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:14.048 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:14.048 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:14.049 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:14.049 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:14.050 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:14.050 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:14.050 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:14.051 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:14.051 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:14.051 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:18.048 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:18.051 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:18.051 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:18.052 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:18.052 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:18.053 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:18.053 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:18.054 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:18.054 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:18.055 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:18.055 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:18.055 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:18.056 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:18.056 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:18.060 7fd7811b0e00 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:11:21.795 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:11:22.052 7fd7811b0400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:22.054 7fd7811b0400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:22.054 7fd7811b0400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:22.055 7fd7811b0400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:22.055 7fd7811b0400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:22.055 7fd7811b0400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:22.056 7fd7811b0400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:22.056 7fd7811b0400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:22.057 7fd7811b0400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:22.057 7fd7811b0400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:22.057 7fd7811b0400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:22.058 7fd7811b0400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:22.058 7fd7811b0400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:22.058 7fd7811b0400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:22.341 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:22.342 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19577232 disk_space_available_1k = 61513252 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:11:22.342 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:22.760 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:11:23.979 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:11:26.056 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:26.059 7fd7811b00e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:26.060 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:26.060 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:26.060 7fd7811b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:26.061 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:26.061 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:26.062 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:26.062 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:26.062 7fd7811b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:26.063 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:26.063 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:26.063 7fd7811b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:26.063 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:27.036 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19577356, partition_space_available_1k=61513128, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1304944, cpu_share=0.05, mem_res=1293136, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46496, cpu_share=0.04, mem_res=43168, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9383992, free=19752696, buffers=720872, cached=2807548, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.7, five_min_avg=1.64, fifteen_min_avg=1.75, cpu_share=1.62, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:11:27.036 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:27.036 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19577356 partition_space_available_1k = 61513128 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1304944 cpu_share = 0.05 mem_res = 1293136 >>contrail-database-nodemgr : << mem_virt = 46496 cpu_share = 0.04 mem_res = 43168 >> } system_mem_usage = << total = 32665108 used = 9383992 free = 19752696 buffers = 720872 cached = 2807548 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.7 five_min_avg = 1.64 fifteen_min_avg = 1.75 cpu_share = 1.62 node_type = database-node >> >> 09/01/2025 02:11:27.037 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:30.063 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:30.065 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:30.065 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:30.065 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:30.065 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:30.066 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:30.066 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:30.066 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:30.066 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:30.067 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:30.067 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:30.067 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:30.067 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:30.067 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:30.962 7fd7811cd7c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:11:34.068 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:34.070 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:34.071 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:34.071 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:34.072 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:34.072 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:34.072 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:34.073 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:34.073 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:34.073 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:34.074 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:34.074 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:34.074 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:34.074 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:38.072 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:38.075 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:38.076 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:38.076 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:38.077 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:38.077 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:38.078 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:38.079 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:38.079 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:38.079 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:38.079 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:38.080 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:38.080 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:38.080 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:42.077 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:42.080 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:42.080 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:42.081 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:42.081 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:42.081 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:42.082 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:42.083 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:42.083 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:42.084 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:42.084 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:42.084 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:42.084 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:42.085 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:43.871 7fd7811cda40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:11:46.082 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:46.083 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:46.084 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:46.084 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:46.084 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:46.084 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:46.084 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:46.084 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:46.084 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:46.084 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:46.084 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:46.084 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:46.085 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:46.085 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:50.087 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:50.089 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:50.090 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:50.090 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:50.090 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:50.090 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:50.091 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:50.091 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:50.092 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:50.092 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:50.092 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:50.092 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:50.093 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:50.093 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:54.091 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:54.093 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:54.093 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:54.094 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:54.094 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:54.094 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:54.094 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:54.095 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:54.095 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:54.096 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:54.096 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:54.096 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:54.097 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:54.097 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:56.853 7fd7811b0860 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:11:58.095 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:58.098 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:58.098 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:58.098 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:58.099 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:58.099 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:58.099 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:58.100 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:58.100 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:58.101 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:58.101 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:58.101 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:58.102 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:58.102 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:02.103 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:02.104 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:02.104 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:02.104 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:02.104 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:02.104 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:02.104 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:02.104 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:02.105 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:02.105 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:02.105 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:02.105 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:02.105 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:02.105 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:06.106 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:06.108 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:06.108 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:06.109 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:06.109 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:06.109 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:06.110 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:06.111 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:06.111 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:06.111 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:06.111 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:06.112 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:06.112 7fd7811b0860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:06.112 7fd7811b0860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:09.730 7fd7811b0d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:12:10.110 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:10.112 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:10.112 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:10.112 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:10.113 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:10.113 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:10.113 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:10.114 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:10.115 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:10.115 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:10.115 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:10.116 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:10.116 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:10.117 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:14.115 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:14.117 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:14.118 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:14.118 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:14.119 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:14.119 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:14.119 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:14.120 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:14.121 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:14.122 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:14.122 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:14.122 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:14.123 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:14.123 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:18.122 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:18.123 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:18.123 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:18.123 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:18.124 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:18.124 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:18.124 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:18.124 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:18.124 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:18.124 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:18.124 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:18.124 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:18.124 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:18.125 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:21.782 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:12:22.123 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:22.131 7fd7811b00e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:22.131 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:22.131 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:22.132 7fd7811b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:22.132 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:22.132 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:22.133 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:22.133 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:22.133 7fd7811b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:22.133 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:22.134 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:22.134 7fd7811b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:22.134 7fd7811b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:22.446 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:22.447 7fd7811b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19582672 disk_space_available_1k = 61507812 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:12:22.447 7fd7811b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:22.819 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:12:22.871 7fd7811b0180 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:12:23.956 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:12:26.133 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:26.137 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:26.138 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:26.138 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:26.138 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:26.139 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:26.139 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:26.140 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:26.140 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:26.141 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:26.141 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:26.142 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:26.142 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:26.143 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:27.027 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19580540, partition_space_available_1k=61509944, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1305052, cpu_share=0.05, mem_res=1293212, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46604, cpu_share=0.04, mem_res=43188, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9389124, free=19743956, buffers=720944, cached=2811084, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.61, five_min_avg=1.65, fifteen_min_avg=1.74, cpu_share=2.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:12:27.028 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:27.029 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19580540 partition_space_available_1k = 61509944 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1305052 cpu_share = 0.05 mem_res = 1293212 >>contrail-database-nodemgr : << mem_virt = 46604 cpu_share = 0.04 mem_res = 43188 >> } system_mem_usage = << total = 32665108 used = 9389124 free = 19743956 buffers = 720944 cached = 2811084 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.61 five_min_avg = 1.65 fifteen_min_avg = 1.74 cpu_share = 2.0 node_type = database-node >> >> 09/01/2025 02:12:27.029 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:30.140 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:30.141 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:30.141 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:30.141 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:30.141 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:30.141 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:30.141 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:30.142 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:30.142 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:30.142 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:30.142 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:30.142 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:30.142 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:30.142 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:34.146 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:34.148 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:34.148 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:34.148 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:34.149 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:34.149 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:34.150 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:34.150 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:34.151 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:34.151 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:34.151 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:34.152 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:34.152 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:34.152 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:35.619 7fd7811cdc20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:12:38.150 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:38.155 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:38.155 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:38.155 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:38.156 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:38.156 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:38.156 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:38.157 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:38.157 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:38.157 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:38.157 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:38.157 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:38.157 7fd7811b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:38.158 7fd7811b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:42.159 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:42.160 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:42.161 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:42.162 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:42.162 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:42.162 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:42.163 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:42.164 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:42.164 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:42.165 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:42.165 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:42.166 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:42.166 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:42.166 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:46.162 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:46.164 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:46.164 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:46.165 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:46.165 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:46.165 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:46.166 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:46.166 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:46.167 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:46.167 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:46.167 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:46.168 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:46.168 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:46.168 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:48.651 7fd7811cd7c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:12:50.166 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:50.168 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:50.168 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:50.168 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:50.168 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:50.168 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:50.168 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:50.169 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:50.169 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:50.169 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:50.169 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:50.169 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:50.170 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:50.170 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:54.168 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:54.171 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:54.171 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:54.172 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:54.172 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:54.172 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:54.173 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:54.174 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:54.174 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:54.175 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:54.175 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:54.175 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:54.176 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:54.176 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:58.172 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:58.173 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:58.173 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:58.173 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:58.173 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:58.173 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:58.173 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:58.174 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:58.174 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:58.174 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:58.174 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:58.174 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:58.174 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:58.174 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:01.590 7fd7811b0e00 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:13:02.174 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:02.176 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:02.177 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:02.177 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:02.178 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:02.178 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:02.178 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:02.179 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:02.180 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:02.180 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:02.180 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:02.181 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:02.181 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:02.181 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:06.181 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:06.183 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:06.184 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:06.184 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:06.185 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:06.185 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:06.185 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:06.186 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:06.186 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:06.187 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:06.187 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:06.187 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:06.188 7fd7811b0e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:06.188 7fd7811b0e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:10.186 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:10.188 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:10.189 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:10.189 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:10.189 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:10.190 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:10.190 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:10.191 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:10.191 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:10.191 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:10.192 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:10.192 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:10.192 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:10.193 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:14.190 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:14.192 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:14.193 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:14.193 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:14.193 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:14.194 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:14.194 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:14.195 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:14.195 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:14.195 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:14.195 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:14.196 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:14.196 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:14.196 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:14.456 7fd7811cdea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:13:18.197 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:18.199 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:18.199 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:18.199 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:18.200 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:18.200 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:18.200 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:18.201 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:18.202 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:18.202 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:18.202 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:18.202 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:18.203 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:18.203 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:21.664 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:13:22.199 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:22.200 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:22.200 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:22.200 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:22.201 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:22.201 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:22.201 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:22.201 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:22.201 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:22.202 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:22.202 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:22.202 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:22.202 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:22.202 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:22.276 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:22.276 7fd7811b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19583736 disk_space_available_1k = 61506748 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:13:22.276 7fd7811b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:22.618 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:13:23.753 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:13:26.202 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:26.205 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:26.206 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:26.206 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:26.206 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:26.206 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:26.207 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:26.207 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:26.208 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:26.208 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:26.208 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:26.208 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:26.209 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:26.209 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:26.848 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19583776, partition_space_available_1k=61506708, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1305164, cpu_share=0.05, mem_res=1293292, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46692, cpu_share=0.04, mem_res=43188, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9415056, free=19713684, buffers=721020, cached=2815348, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.33, five_min_avg=1.58, fifteen_min_avg=1.71, cpu_share=1.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:13:26.848 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:26.848 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19583776 partition_space_available_1k = 61506708 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1305164 cpu_share = 0.05 mem_res = 1293292 >>contrail-database-nodemgr : << mem_virt = 46692 cpu_share = 0.04 mem_res = 43188 >> } system_mem_usage = << total = 32665108 used = 9415056 free = 19713684 buffers = 721020 cached = 2815348 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.33 five_min_avg = 1.58 fifteen_min_avg = 1.71 cpu_share = 1.75 node_type = database-node >> >> 09/01/2025 02:13:26.848 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:27.814 7fd7811b0d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:13:30.209 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:30.212 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:30.212 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:30.213 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:30.213 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:30.214 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:30.214 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:30.215 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:30.215 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:30.216 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:30.216 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:30.217 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:30.217 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:30.217 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:34.216 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:34.219 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:34.219 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:34.219 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:34.220 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:34.220 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:34.220 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:34.221 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:34.221 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:34.222 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:34.222 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:34.222 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:34.222 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:34.223 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:38.222 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:38.224 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:38.224 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:38.224 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:38.225 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:38.225 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:38.225 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:38.226 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:38.226 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:38.227 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:38.227 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:38.227 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:38.227 7fd7811b0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:38.228 7fd7811b0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:40.688 7fd7811b0cc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:13:42.225 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:42.227 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:42.228 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:42.228 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:42.229 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:42.229 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:42.229 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:42.231 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:42.231 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:42.231 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:42.232 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:42.232 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:42.232 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:42.233 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:46.230 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:46.232 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:46.233 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:46.233 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:46.234 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:46.234 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:46.234 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:46.235 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:46.235 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:46.236 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:46.236 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:46.236 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:46.237 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:46.237 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:50.237 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:50.239 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:50.239 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:50.240 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:50.240 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:50.240 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:50.240 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:50.241 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:50.242 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:50.242 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:50.242 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:50.243 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:50.243 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:50.243 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:53.441 7fd7811b0cc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:13:54.241 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:54.243 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:54.243 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:54.244 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:54.244 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:54.244 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:54.245 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:54.246 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:54.246 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:54.246 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:54.247 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:54.247 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:54.247 7fd7811b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:54.248 7fd7811b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:58.244 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:58.248 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:58.248 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:58.249 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:58.249 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:58.250 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:58.250 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:58.251 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:58.252 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:58.252 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:58.253 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:58.253 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:58.253 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:58.254 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:02.249 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:02.250 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:02.251 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:02.251 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:02.251 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:02.251 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:02.251 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:02.251 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:02.251 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:02.252 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:02.252 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:02.252 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:02.252 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:02.252 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:06.150 7fd781158ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:14:06.251 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:06.253 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:06.253 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:06.254 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:06.254 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:06.254 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:06.254 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:06.255 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:06.255 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:06.255 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:06.256 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:06.256 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:06.256 7fd781158ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:06.256 7fd781158ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:10.254 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:10.255 7fd7811b0220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:10.256 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:10.256 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:10.256 7fd7811b0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:10.256 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:10.256 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:10.257 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:10.257 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:10.257 7fd7811b0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:10.258 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:10.258 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:10.258 7fd7811b0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:10.258 7fd7811b0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:14.260 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:14.262 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:14.262 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:14.263 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:14.263 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:14.263 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:14.263 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:14.264 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:14.264 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:14.264 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:14.265 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:14.265 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:14.265 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:14.265 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:18.267 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:18.269 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:18.270 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:18.270 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:18.271 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:18.271 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:18.271 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:18.272 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:18.272 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:18.273 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:18.273 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:18.274 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:18.274 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:18.275 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:19.035 7fd7811cd7c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:14:21.622 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:14:22.266 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:22.266 7fd7811cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19587204 disk_space_available_1k = 61503280 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:14:22.266 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:22.270 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:22.272 7fd7811cde00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:22.273 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:22.273 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:22.273 7fd7811cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:22.274 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:22.274 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:22.275 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:22.275 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:22.276 7fd7811cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:22.276 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:22.276 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:22.277 7fd7811cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:22.277 7fd7811cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:22.704 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:14:23.979 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:14:26.274 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:26.278 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:26.278 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:26.279 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:26.279 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:26.279 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:26.280 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:26.281 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:26.281 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:26.281 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:26.282 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:26.282 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:26.283 7fd7811cdc20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:26.283 7fd7811cdc20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:27.038 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19587256, partition_space_available_1k=61503228, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1305424, cpu_share=0.06, mem_res=1293364, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46796, cpu_share=0.03, mem_res=43188, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9418852, free=19701904, buffers=721084, cached=2823268, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.08, five_min_avg=1.47, fifteen_min_avg=1.67, cpu_share=1.62, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:14:27.039 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:27.040 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19587256 partition_space_available_1k = 61503228 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1305424 cpu_share = 0.06 mem_res = 1293364 >>contrail-database-nodemgr : << mem_virt = 46796 cpu_share = 0.03 mem_res = 43188 >> } system_mem_usage = << total = 32665108 used = 9418852 free = 19701904 buffers = 721084 cached = 2823268 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.08 five_min_avg = 1.47 fifteen_min_avg = 1.67 cpu_share = 1.62 node_type = database-node >> >> 09/01/2025 02:14:27.040 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:30.279 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:30.280 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:30.281 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:30.281 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:30.281 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:30.282 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:30.282 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:30.283 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:30.283 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:30.284 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:30.284 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:30.284 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:30.285 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:30.285 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:32.098 7fd781158f40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:14:34.281 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:34.283 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:34.283 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:34.283 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:34.283 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:34.283 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:34.283 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:34.284 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:34.284 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:34.284 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:34.284 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:34.284 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:34.284 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:34.284 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:38.285 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:38.287 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:38.288 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:38.288 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:38.289 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:38.289 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:38.289 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:38.290 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:38.290 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:38.291 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:38.291 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:38.291 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:38.292 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:38.292 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:42.289 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:42.291 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:42.291 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:42.292 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:42.292 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:42.292 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:42.292 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:42.293 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:42.293 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:42.294 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:42.294 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:42.294 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:42.295 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:42.295 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:44.962 7fd781158f40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:14:46.293 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:46.294 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:46.294 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:46.294 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:46.295 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:46.295 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:46.295 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:46.295 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:46.295 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:46.295 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:46.296 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:46.296 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:46.296 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:46.296 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:50.298 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:50.300 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:50.301 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:50.301 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:50.301 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:50.301 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:50.302 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:50.302 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:50.303 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:50.303 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:50.303 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:50.304 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:50.304 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:50.304 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:54.302 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:54.303 7fd781158f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:54.304 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:54.304 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:54.304 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:54.304 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:54.305 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:54.305 7fd781158f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:54.305 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:54.306 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:54.306 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:54.306 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:54.306 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:54.307 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:57.916 7fd781158c20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:14:58.305 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:58.308 7fd781158c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:58.308 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:58.309 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:58.309 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:58.309 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:58.310 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:58.311 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:58.311 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:58.311 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:58.312 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:58.312 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:58.312 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:58.312 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:02.309 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:02.311 7fd781158c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:02.311 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:02.312 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:02.312 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:02.312 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:02.312 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:02.313 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:02.313 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:02.313 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:02.314 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:02.314 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:02.314 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:02.314 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:06.316 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:06.318 7fd781158c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:06.319 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:06.319 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:06.320 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:06.320 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:06.320 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:06.321 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:06.321 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:06.321 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:06.321 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:06.322 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:06.322 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:06.322 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:10.322 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:10.324 7fd781158c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:10.324 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:10.325 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:10.325 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:10.326 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:10.326 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:10.327 7fd781158c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:10.327 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:10.328 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:10.328 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:10.328 7fd781158c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:10.329 7fd781158c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:10.329 7fd781158c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:10.976 7fd781158c20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:15:14.328 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:14.330 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:14.331 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:14.331 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:14.332 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:14.332 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:14.332 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:14.333 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:14.333 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:14.334 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:14.334 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:14.334 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:14.334 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:14.335 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:18.335 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:18.338 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:18.339 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:18.339 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:18.339 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:18.340 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:18.340 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:18.341 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:18.341 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:18.341 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:18.342 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:18.342 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:18.342 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:18.343 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:21.737 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:15:22.339 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:22.347 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:22.347 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:22.348 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:22.348 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:22.348 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:22.349 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:22.350 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:22.350 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:22.351 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:22.351 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:22.351 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:22.351 7fd7811b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:22.352 7fd7811b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:22.585 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:22.586 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19598652 disk_space_available_1k = 61491832 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:15:22.586 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:22.990 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:15:23.821 7fd7811b0720 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:15:24.341 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:15:26.349 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:26.352 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:26.352 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:26.353 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:26.353 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:26.354 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:26.354 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:26.355 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:26.356 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:26.356 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:26.357 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:26.357 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:26.358 7fd7811cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:26.358 7fd7811cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:27.392 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19598712, partition_space_available_1k=61491772, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1305424, cpu_share=0.05, mem_res=1293364, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=47012, cpu_share=0.04, mem_res=43188, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9396504, free=19725076, buffers=721112, cached=2822416, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.42, five_min_avg=1.52, fifteen_min_avg=1.67, cpu_share=1.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:15:27.392 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:27.393 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19598712 partition_space_available_1k = 61491772 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1305424 cpu_share = 0.05 mem_res = 1293364 >>contrail-database-nodemgr : << mem_virt = 47012 cpu_share = 0.04 mem_res = 43188 >> } system_mem_usage = << total = 32665108 used = 9396504 free = 19725076 buffers = 721112 cached = 2822416 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.42 five_min_avg = 1.52 fifteen_min_avg = 1.67 cpu_share = 1.75 node_type = database-node >> >> 09/01/2025 02:15:27.394 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:30.353 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:30.355 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:30.355 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:30.355 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:30.355 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:30.355 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:30.356 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:30.356 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:30.356 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:30.357 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:30.357 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:30.357 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:30.357 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:30.357 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:34.360 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:34.363 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:34.363 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:34.364 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:34.365 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:34.365 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:34.366 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:34.367 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:34.367 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:34.368 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:34.368 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:34.369 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:34.369 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:34.370 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:37.076 7fd7811cd5e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:15:38.365 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:38.367 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:38.369 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:38.369 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:38.370 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:38.370 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:38.370 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:38.371 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:38.371 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:38.372 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:38.372 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:38.372 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:38.372 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:38.373 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:39.828 7fd7811cd5e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:15:42.368 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:42.370 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:42.370 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:42.370 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:42.370 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:42.371 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:42.371 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:42.371 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:42.372 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:42.372 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:42.372 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:42.372 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:42.372 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:42.373 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:42.733 7fd7811cdea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:15:46.372 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:46.374 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:46.374 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:46.375 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:46.375 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:46.376 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:46.376 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:46.377 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:46.377 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:46.378 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:46.378 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:46.378 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:46.379 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:46.379 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:50.379 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:50.381 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:50.382 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:50.382 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:50.383 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:50.383 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:50.383 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:50.384 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:50.385 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:50.385 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:50.385 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:50.386 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:50.386 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:50.386 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:54.383 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:54.384 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:54.385 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:54.385 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:54.385 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:54.385 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:54.385 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:54.386 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:54.386 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:54.386 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:54.387 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:54.387 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:54.387 7fd7811cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:54.387 7fd7811cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:58.386 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:58.388 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:58.388 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:58.389 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:58.389 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:58.389 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:58.389 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:58.390 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:58.391 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:58.391 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:58.391 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:58.391 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:58.392 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:58.392 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:02.389 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:02.390 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:02.390 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:02.391 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:02.391 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:02.391 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:02.391 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:02.391 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:02.391 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:02.391 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:02.391 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:02.392 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:02.392 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:02.392 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:06.394 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:06.396 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:06.396 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:06.397 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:06.397 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:06.397 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:06.398 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:06.399 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:06.399 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:06.399 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:06.399 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:06.400 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:06.400 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:06.400 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:10.397 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:10.398 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:10.398 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:10.399 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:10.399 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:10.399 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:10.399 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:10.399 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:10.399 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:10.399 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:10.399 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:10.400 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:10.400 7fd7811cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:10.400 7fd7811cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:14.400 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:14.403 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:14.403 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:14.404 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:14.404 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:14.404 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:14.405 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:14.406 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:14.406 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:14.406 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:14.406 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:14.407 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:14.407 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:14.407 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:17.098 7fd7811cd400 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:16:18.404 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:18.407 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:18.407 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:18.408 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:18.409 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:18.409 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:18.409 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:18.410 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:18.411 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:18.411 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:18.411 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:18.412 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:18.412 7fd7811cd400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:18.413 7fd7811cd400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:21.674 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:16:22.355 7fd781158f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:22.355 7fd781158f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. database_usage = [ << disk_space_used_1k = 19605332 disk_space_available_1k = 61485152 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:16:22.355 7fd781158f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:22.408 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:22.411 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:22.411 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:22.411 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:22.411 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:22.411 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:22.411 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:22.412 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:22.412 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:22.412 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:22.412 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:22.412 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:22.412 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:22.412 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:22.714 7fd783f1b0c0 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:16:23.857 7fd783f1b0c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:16:26.413 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:26.416 7fd7811b02c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:26.416 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:26.416 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:26.417 7fd7811b02c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:26.417 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:26.417 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:26.418 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:26.418 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:26.418 7fd7811b02c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:26.418 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:26.419 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:26.419 7fd7811b02c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:26.419 7fd7811b02c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:26.903 7fd783f1b0c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3992-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19608060, partition_space_available_1k=61482424, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1305492, cpu_share=0.05, mem_res=1293428, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46972, cpu_share=0.03, mem_res=43188, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665108, used=9406228, free=19506132, buffers=721196, cached=3031552, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.59, five_min_avg=1.52, fifteen_min_avg=1.66, cpu_share=2.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:16:26.904 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:26.905 7fd7811cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19608060 partition_space_available_1k = 61482424 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1305492 cpu_share = 0.05 mem_res = 1293428 >>contrail-database-nodemgr : << mem_virt = 46972 cpu_share = 0.03 mem_res = 43188 >> } system_mem_usage = << total = 32665108 used = 9406228 free = 19506132 buffers = 721196 cached = 3031552 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.59 five_min_avg = 1.52 fifteen_min_avg = 1.66 cpu_share = 2.25 node_type = database-node >> >> 09/01/2025 02:16:26.905 7fd7811cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:30.418 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:30.420 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:30.421 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:30.421 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:30.422 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:30.422 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:30.422 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:30.423 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:30.423 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:30.424 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:30.424 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:30.424 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:30.425 7fd7811cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3992-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691901022458 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:30.425 7fd7811cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]