09/01/2025 01:50:58.789 7f0dd425b280 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/01/2025 01:50:58.789 7f0dd425b280 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/01/2025 01:50:58.789 7f0dd425b280 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/01/2025 01:50:58.798 7f0dd13c84a0 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 09/01/2025 01:50:58.799 7f0dd13c84a0 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 09/01/2025 01:50:58.800 7f0dd13c8860 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 09/01/2025 01:50:58.800 7f0dd13c8860 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:50:58.801 7f0dd13c8860 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/01/2025 01:50:58.801 7f0dd13c8860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/01/2025 01:50:58.801 7f0dd13c8860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:58.801 7f0dd13c8860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:50:58.801 7f0dd13c8860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:58.801 7f0dd13c8860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:58.801 7f0dd13c8860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/01/2025 01:50:58.801 7f0dd13c8860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:59.118 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 102100800431694675582377446962655250676636844892861728646012027667195731348230, 'start': '1756691456000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 110212463790988970921635985444523713893101715041124700875401898537940162771173, 'start': '1756691458000000', 'statename': 'PROCESS_STATE_BACKOFF'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 35457963262649879895049066676671037809955932247073362510994505114634130103222, 'start': '1756691457000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 09/01/2025 01:50:59.194 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:59.195 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = package contrail-lib is not installed installed_package_version = package contrail-lib is not installed >> 09/01/2025 01:50:59.195 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:59.195 7f0dd425b280 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3994-1.', _node_type='Database', _instance_id='0', _timestamp=1756691459195850, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:50:59.196 7f0dd425b280 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3994-1.', _node_type='Database', _instance_id='0', _timestamp=1756691459196390, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691456000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_BACKOFF', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691458000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691457000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:50:59.199 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:59.199 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:50:59.199 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:59.199 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:59.199 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691456000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_BACKOFF start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691458000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691457000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 09/01/2025 01:50:59.199 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:59.565 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 01:50:59.624 7f0dd14c3ea0 [contrail-database-nodemgr] [DEBUG]: DBG: event_process_state:contrail-query-engine,group:contrail-query-engine,state:PROCESS_STATE_RUNNING 09/01/2025 01:50:59.625 7f0dd14c3ea0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3994-1.', _node_type='Database', _instance_id='0', _timestamp=1756691459625120, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691456000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=2, stop_count=0, exit_count=0, last_start_time='1756691459625065', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691457000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:50:59.626 7f0dd14d49a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:59.626 7f0dd14d49a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691456000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 2 stop_count = 0 exit_count = 0 last_start_time = 1756691459625065 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691457000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 09/01/2025 01:50:59.626 7f0dd14d49a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:59.912 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 09/01/2025 01:50:59.913 7f0dd14d49a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:59.914 7f0dd14d49a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 16868384 disk_space_available_1k = 64222100 analytics_db_size_1k = 108 >>, ] >> 09/01/2025 01:50:59.914 7f0dd14d49a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:00.107 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 09/01/2025 01:51:00.107 7f0dd425b280 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3994-1.', _node_type='Database', _instance_id='0', _timestamp=1756691460107469, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:51:00.108 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:00.109 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:51:00.109 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:01.147 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 01:51:02.254 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7200 repair -pr ContrailAnalyticsCql'. Error is: Result: 1 09/01/2025 01:51:02.801 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:02.803 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:02.803 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:02.803 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:02.803 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:02.803 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:02.803 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:02.804 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:02.804 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:02.804 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:02.804 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:02.804 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:02.804 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:02.804 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:05.287 7f0dd425b280 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3994-1.', _node_type='Database', _instance_id='0', _timestamp=1756691465287236, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691456000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=2, stop_count=0, exit_count=0, last_start_time='1756691459625065', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691457000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:51:05.287 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17014452, partition_space_available_1k=64076032, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1265552, cpu_share=0.0, mem_res=1255480, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=0, cpu_share=0.0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52088, cpu_share=0.0, mem_res=42664, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=7838916, free=6984432, buffers=634764, cached=17207008, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.49, five_min_avg=2.16, fifteen_min_avg=1.32, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 01:51:05.287 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:05.288 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691456000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 2 stop_count = 0 exit_count = 0 last_start_time = 1756691459625065 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691457000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 09/01/2025 01:51:05.288 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:05.288 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:05.288 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 17014452 partition_space_available_1k = 64076032 percentage_partition_space_used = 21 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1265552 cpu_share = 0.0 mem_res = 1255480 >>contrail-query-engine : << mem_virt = 0 cpu_share = 0.0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 52088 cpu_share = 0.0 mem_res = 42664 >> } system_mem_usage = << total = 32665120 used = 7838916 free = 6984432 buffers = 634764 cached = 17207008 node_type = database-node >> system_cpu_usage = << one_min_avg = 3.49 five_min_avg = 2.16 fifteen_min_avg = 1.32 cpu_share = 0 node_type = database-node >> >> 09/01/2025 01:51:05.288 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:06.803 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:06.805 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:06.805 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:06.805 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:06.805 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:06.805 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:06.805 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:06.806 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:06.806 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:06.806 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:06.806 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:06.806 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:06.806 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:06.806 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:10.808 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:10.809 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:10.809 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:10.809 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:10.809 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:10.809 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:10.809 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:10.810 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:10.810 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:10.810 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:10.810 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:10.810 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:10.810 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:10.810 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:14.732 7f0dd14c3ea0 [contrail-database-nodemgr] [DEBUG]: DBG: event_process_state:contrail-query-engine,group:contrail-query-engine,state:PROCESS_STATE_BACKOFF 09/01/2025 01:51:14.810 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:14.811 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:14.811 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:14.811 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:14.811 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:14.811 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:14.811 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:14.811 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:14.812 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:14.812 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:14.812 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:14.812 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:14.812 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:14.812 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:18.815 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:18.816 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:18.817 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:18.817 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:18.817 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:18.817 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:18.817 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:18.817 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:18.817 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:18.817 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:18.818 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:18.818 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:18.818 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:18.818 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:22.820 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:22.821 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:22.821 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:22.821 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:22.821 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:22.821 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:22.822 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:22.822 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:22.822 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:22.822 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:22.822 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:22.822 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:22.822 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:22.822 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:26.824 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:26.825 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:26.825 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:26.825 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:26.825 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:26.825 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:26.825 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:26.826 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:26.826 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:26.826 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:26.826 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:26.826 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:26.826 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:26.826 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:30.826 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:30.827 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:30.827 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:30.827 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:30.827 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:30.828 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:30.828 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:30.828 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:30.828 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:30.828 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:30.828 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:30.828 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:30.828 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:30.829 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:34.832 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:34.833 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:34.833 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:34.833 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:34.833 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:34.833 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:34.833 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:34.834 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:34.834 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:34.834 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:34.834 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:34.834 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:34.834 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:34.834 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:38.837 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:38.838 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:38.838 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:38.838 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:38.838 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:38.838 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:38.838 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:38.839 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:38.839 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:38.839 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:38.839 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:38.839 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:38.839 7f0dd14d4680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:38.839 7f0dd14d4680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:42.842 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:42.843 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:42.843 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:42.843 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:42.843 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:42.843 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:42.843 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:42.844 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:42.844 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:42.844 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:42.844 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:42.844 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:42.844 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:42.844 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:46.844 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:46.845 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:46.846 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:46.846 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:46.846 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:46.846 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:46.846 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:46.846 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:46.846 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:46.847 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:46.847 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:46.847 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:46.847 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:46.847 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:50.848 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:50.849 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:50.849 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:50.850 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:50.850 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:50.850 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:50.850 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:50.850 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:50.850 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:50.851 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:50.851 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:50.851 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:50.851 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:50.851 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:54.850 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:54.851 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:54.851 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:54.851 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:54.851 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:54.851 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:54.851 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:54.852 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:54.852 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:54.852 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:54.852 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:54.852 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:54.852 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:54.852 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:58.852 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:58.853 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:58.853 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:58.853 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:58.853 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:58.853 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:58.854 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:58.854 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:58.854 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:58.854 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:58.854 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:58.854 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:58.854 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:51:58.854 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:59.435 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 01:51:59.776 7f0dd14d4220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:59.777 7f0dd14d4220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 18468928 disk_space_available_1k = 62621556 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 01:51:59.777 7f0dd14d4220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:59.946 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 01:51:59.946 7f0dd425b280 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3994-1.', _node_type='Database', _instance_id='0', _timestamp=1756691519946598, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.33:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:51:59.947 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:59.948 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:59.948 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:00.774 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 01:52:02.855 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:02.856 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:02.856 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:02.856 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:02.856 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:02.856 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:02.857 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:02.857 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:02.857 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:02.857 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:02.857 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:02.857 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:02.857 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:02.857 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:03.800 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18592108, partition_space_available_1k=62498376, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1305316, cpu_share=0.09, mem_res=1294612, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52548, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9233436, free=3808388, buffers=662692, cached=18960604, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.0, five_min_avg=2.24, fifteen_min_avg=1.4, cpu_share=3.62, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 01:52:03.801 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:03.801 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 18592108 partition_space_available_1k = 62498376 percentage_partition_space_used = 23 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1305316 cpu_share = 0.09 mem_res = 1294612 >>contrail-database-nodemgr : << mem_virt = 52548 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 9233436 free = 3808388 buffers = 662692 cached = 18960604 node_type = database-node >> system_cpu_usage = << one_min_avg = 3.0 five_min_avg = 2.24 fifteen_min_avg = 1.4 cpu_share = 3.62 node_type = database-node >> >> 09/01/2025 01:52:03.801 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:06.857 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:06.858 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:06.858 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:06.858 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:06.858 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:06.858 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:06.858 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:06.859 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:06.859 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:06.859 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:06.859 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:06.859 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:06.859 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:06.859 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:10.861 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:10.862 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:10.862 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:10.863 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:10.863 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:10.863 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:10.863 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:10.863 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:10.863 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:10.863 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:10.863 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:10.863 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:10.864 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:10.864 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:14.865 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:14.866 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:14.866 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:14.866 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:14.866 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:14.866 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:14.866 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:14.867 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:14.867 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:14.867 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:14.867 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:14.867 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:14.867 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:14.867 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:18.869 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:18.870 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:18.870 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:18.870 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:18.870 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:18.871 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:18.871 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:18.871 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:18.872 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:18.872 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:18.872 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:18.872 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:18.872 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:18.872 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:22.872 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:22.873 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:22.874 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:22.874 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:22.874 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:22.874 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:22.874 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:22.874 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:22.874 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:22.875 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:22.875 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:22.875 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:22.875 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:22.875 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:26.878 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:26.879 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:26.880 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:26.880 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:26.880 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:26.880 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:26.880 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:26.880 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:26.880 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:26.880 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:26.881 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:26.881 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:26.881 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:26.881 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:30.880 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:30.881 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:30.882 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:30.882 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:30.882 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:30.882 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:30.882 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:30.882 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:30.882 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:30.882 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:30.882 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:30.883 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:30.883 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:30.883 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:34.886 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:34.887 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:34.887 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:34.888 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:34.888 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:34.888 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:34.888 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:34.888 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:34.888 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:34.889 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:34.889 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:34.889 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:34.889 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:34.889 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:38.892 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:38.893 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:38.893 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:38.893 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:38.893 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:38.893 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:38.894 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:38.894 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:38.894 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:38.894 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:38.894 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:38.894 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:38.894 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:38.894 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:42.896 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:42.897 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:42.897 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:42.897 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:42.897 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:42.897 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:42.898 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:42.898 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:42.898 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:42.898 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:42.898 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:42.898 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:42.898 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:42.898 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:46.899 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:46.900 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:46.900 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:46.900 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:46.900 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:46.900 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:46.900 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:46.901 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:46.901 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:46.901 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:46.901 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:46.901 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:46.901 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:46.901 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:50.901 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:50.902 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:50.903 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:50.903 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:50.903 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:50.903 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:50.903 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:50.903 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:50.903 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:50.903 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:50.904 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:50.904 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:50.904 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:50.904 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:54.904 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:54.905 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:54.905 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:54.905 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:54.905 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:54.906 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:54.906 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:54.906 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:54.906 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:54.906 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:54.906 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:54.906 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:54.906 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:54.907 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:58.908 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:58.909 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:58.909 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:58.910 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:58.910 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:58.910 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:58.910 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:58.911 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:58.911 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:58.911 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:58.911 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:58.912 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:58.912 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:52:58.912 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:59.442 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 01:52:59.799 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:59.799 7f0dd14d4c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19480716 disk_space_available_1k = 61609768 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 01:52:59.799 7f0dd14d4c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:59.964 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 01:53:00.988 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 01:53:02.911 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:02.913 7f0dd14d4900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:02.913 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:02.913 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:02.913 7f0dd14d4900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:02.913 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:02.913 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:02.914 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:02.914 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:02.914 7f0dd14d4900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:02.914 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:02.914 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:02.914 7f0dd14d4900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:02.914 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:04.020 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19482752, partition_space_available_1k=61607732, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1301024, cpu_share=0.03, mem_res=1292528, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45248, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=8736560, free=20594904, buffers=682500, cached=2651156, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.02, five_min_avg=2.41, fifteen_min_avg=1.52, cpu_share=4.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 01:53:04.020 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:04.021 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19482752 partition_space_available_1k = 61607732 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1301024 cpu_share = 0.03 mem_res = 1292528 >>contrail-database-nodemgr : << mem_virt = 45248 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 8736560 free = 20594904 buffers = 682500 cached = 2651156 node_type = database-node >> system_cpu_usage = << one_min_avg = 3.02 five_min_avg = 2.41 fifteen_min_avg = 1.52 cpu_share = 4.25 node_type = database-node >> >> 09/01/2025 01:53:04.021 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:06.916 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:06.917 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:06.917 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:06.917 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:06.918 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:06.918 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:06.918 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:06.918 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:06.918 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:06.918 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:06.918 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:06.918 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:06.918 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:06.919 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:10.922 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:10.923 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:10.924 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:10.924 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:10.924 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:10.924 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:10.924 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:10.924 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:10.924 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:10.925 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:10.925 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:10.925 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:10.925 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:10.925 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:14.928 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:14.929 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:14.929 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:14.929 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:14.930 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:14.930 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:14.930 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:14.930 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:14.930 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:14.930 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:14.930 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:14.930 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:14.931 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:14.931 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:18.932 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:18.933 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:18.933 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:18.933 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:18.933 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:18.933 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:18.933 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:18.934 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:18.934 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:18.934 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:18.934 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:18.934 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:18.934 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:18.934 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:22.935 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:22.936 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:22.936 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:22.936 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:22.936 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:22.936 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:22.936 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:22.937 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:22.937 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:22.937 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:22.937 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:22.937 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:22.937 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:22.937 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:26.940 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:26.941 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:26.942 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:26.942 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:26.942 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:26.942 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:26.942 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:26.942 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:26.942 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:26.943 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:26.943 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:26.943 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:26.943 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:26.943 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:30.944 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:30.945 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:30.945 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:30.945 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:30.945 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:30.945 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:30.945 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:30.946 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:30.946 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:30.946 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:30.946 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:30.946 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:30.946 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:30.946 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:34.949 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:34.950 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:34.950 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:34.950 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:34.950 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:34.951 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:34.951 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:34.951 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:34.951 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:34.951 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:34.951 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:34.951 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:34.951 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:34.951 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:38.955 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:38.956 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:38.956 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:38.956 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:38.956 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:38.956 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:38.956 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:38.957 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:38.957 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:38.957 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:38.957 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:38.957 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:38.957 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:38.957 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:42.961 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:42.962 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:42.962 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:42.962 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:42.962 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:42.962 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:42.962 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:42.963 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:42.963 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:42.963 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:42.963 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:42.963 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:42.963 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:42.963 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:46.963 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:46.964 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:46.964 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:46.964 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:46.964 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:46.965 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:46.965 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:46.965 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:46.965 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:46.965 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:46.965 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:46.965 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:46.965 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:46.965 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:50.968 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:50.969 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:50.969 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:50.969 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:50.969 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:50.969 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:50.969 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:50.970 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:50.970 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:50.970 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:50.970 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:50.970 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:50.970 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:50.970 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:54.973 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:54.974 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:54.974 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:54.974 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:54.975 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:54.975 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:54.975 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:54.975 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:54.975 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:54.975 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:54.975 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:54.975 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:54.975 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:54.976 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:58.979 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:58.980 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:58.980 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:58.980 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:58.980 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:58.980 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:58.980 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:58.981 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:58.981 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:58.981 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:58.981 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:58.981 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:58.981 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:53:58.981 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:59.431 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 01:53:59.772 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:59.773 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19480152 disk_space_available_1k = 61610332 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 01:53:59.773 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:59.937 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 01:54:00.705 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 01:54:02.982 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:02.983 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:02.983 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:02.983 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:02.983 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:02.983 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:02.983 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:02.984 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:02.984 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:02.984 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:02.984 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:02.984 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:02.984 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:02.984 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:03.734 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19480168, partition_space_available_1k=61610316, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1300560, cpu_share=0.03, mem_res=1292612, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45328, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=8610504, free=20813700, buffers=685580, cached=2555336, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.67, five_min_avg=2.16, fifteen_min_avg=1.49, cpu_share=1.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 01:54:03.734 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:03.734 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19480168 partition_space_available_1k = 61610316 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1300560 cpu_share = 0.03 mem_res = 1292612 >>contrail-database-nodemgr : << mem_virt = 45328 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 8610504 free = 20813700 buffers = 685580 cached = 2555336 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.67 five_min_avg = 2.16 fifteen_min_avg = 1.49 cpu_share = 1.5 node_type = database-node >> >> 09/01/2025 01:54:03.734 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:06.987 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:06.988 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:06.988 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:06.988 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:06.989 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:06.989 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:06.989 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:06.989 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:06.989 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:06.989 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:06.989 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:06.989 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:06.990 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:06.990 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:10.993 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:10.994 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:10.994 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:10.994 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:10.994 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:10.995 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:10.995 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:10.995 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:10.995 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:10.995 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:10.995 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:10.995 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:10.995 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:10.995 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:14.999 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:15.000 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:15.000 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:15.000 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:15.000 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:15.001 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:15.001 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:15.001 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:15.001 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:15.001 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:15.001 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:15.001 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:15.001 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:15.001 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:19.003 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:19.005 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:19.005 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:19.005 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:19.005 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:19.005 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:19.005 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:19.006 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:19.006 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:19.006 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:19.006 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:19.006 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:19.006 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:19.006 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:21.487 7f0dd14d45e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:54:23.006 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:23.007 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:23.008 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:23.008 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:23.008 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:23.008 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:23.008 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:23.008 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:23.008 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:23.008 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:23.009 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:23.009 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:23.009 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:23.009 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:27.012 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:27.013 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:27.013 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:27.013 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:27.013 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:27.014 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:27.014 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:27.014 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:27.014 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:27.014 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:27.014 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:27.014 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:27.014 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:27.015 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:31.016 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:31.017 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:31.017 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:31.017 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:31.017 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:31.017 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:31.017 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:31.018 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:31.018 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:31.018 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:31.018 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:31.018 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:31.018 7f0dd14d45e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:31.018 7f0dd14d45e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:33.539 7f0dd14d4cc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:54:35.019 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:35.020 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:35.020 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:35.020 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:35.020 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:35.020 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:35.020 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:35.021 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:35.021 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:35.021 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:35.021 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:35.021 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:35.021 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:35.021 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:39.024 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:39.025 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:39.025 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:39.025 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:39.025 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:39.025 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:39.025 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:39.026 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:39.026 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:39.026 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:39.026 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:39.026 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:39.026 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:39.026 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:43.030 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:43.031 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:43.031 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:43.031 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:43.031 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:43.031 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:43.031 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:43.032 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:43.032 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:43.032 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:43.032 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:43.032 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:43.032 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:43.032 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:45.530 7f0dd14d4cc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:54:47.032 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:47.033 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:47.033 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:47.033 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:47.033 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:47.033 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:47.033 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:47.033 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:47.034 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:47.034 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:47.034 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:47.034 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:47.034 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:47.034 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:51.036 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:51.037 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:51.037 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:51.037 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:51.037 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:51.037 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:51.037 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:51.038 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:51.038 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:51.038 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:51.038 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:51.038 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:51.038 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:51.038 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:55.042 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:55.043 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:55.043 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:55.043 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:55.043 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:55.043 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:55.043 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:55.043 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:55.044 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:55.044 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:55.044 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:55.044 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:55.044 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:55.044 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:57.589 7f0dd14c30e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:54:59.045 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:59.046 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:59.046 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:59.046 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:59.047 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:59.047 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:59.047 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:59.047 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:59.047 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:59.047 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:59.047 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:59.047 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:59.047 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:54:59.047 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:59.448 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 01:54:59.790 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:59.790 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19662532 disk_space_available_1k = 61427952 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 01:54:59.790 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:59.957 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 01:55:00.719 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 01:55:03.048 7f0dd14d49a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:03.049 7f0dd14d49a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:03.049 7f0dd14d49a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:03.050 7f0dd14d49a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:03.050 7f0dd14d49a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:03.050 7f0dd14d49a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:03.050 7f0dd14d49a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:03.050 7f0dd14d49a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:03.050 7f0dd14d49a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:03.050 7f0dd14d49a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:03.051 7f0dd14d49a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:03.051 7f0dd14d49a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:03.051 7f0dd14d49a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:03.051 7f0dd14d49a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:03.747 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19662564, partition_space_available_1k=61427920, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1300672, cpu_share=0.03, mem_res=1292688, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45420, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9316712, free=19917056, buffers=691512, cached=2739840, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.05, five_min_avg=1.9, fifteen_min_avg=1.44, cpu_share=1.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 01:55:03.748 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:03.748 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19662564 partition_space_available_1k = 61427920 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1300672 cpu_share = 0.03 mem_res = 1292688 >>contrail-database-nodemgr : << mem_virt = 45420 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 9316712 free = 19917056 buffers = 691512 cached = 2739840 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.05 five_min_avg = 1.9 fifteen_min_avg = 1.44 cpu_share = 1.25 node_type = database-node >> >> 09/01/2025 01:55:03.748 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:07.052 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:07.053 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:07.053 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:07.053 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:07.053 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:07.053 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:07.053 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:07.054 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:07.054 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:07.054 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:07.054 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:07.054 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:07.054 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:07.054 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:09.664 7f0dd14d4cc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:55:11.055 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:11.056 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:11.057 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:11.057 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:11.057 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:11.057 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:11.057 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:11.057 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:11.057 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:11.057 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:11.058 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:11.058 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:11.058 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:11.058 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:15.060 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:15.061 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:15.062 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:15.062 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:15.062 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:15.062 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:15.062 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:15.062 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:15.062 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:15.063 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:15.063 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:15.063 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:15.063 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:15.063 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:19.064 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:19.065 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:19.065 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:19.065 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:19.065 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:19.065 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:19.065 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:19.066 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:19.066 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:19.066 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:19.066 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:19.066 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:19.066 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:19.066 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:21.986 7f0dd14d4360 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:55:23.067 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:23.068 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:23.068 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:23.068 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:23.069 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:23.069 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:23.069 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:23.069 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:23.069 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:23.069 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:23.069 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:23.069 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:23.069 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:23.070 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:27.070 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:27.071 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:27.071 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:27.071 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:27.071 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:27.071 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:27.071 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:27.072 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:27.072 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:27.072 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:27.072 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:27.072 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:27.072 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:27.072 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:31.076 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:31.077 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:31.077 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:31.077 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:31.077 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:31.078 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:31.078 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:31.078 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:31.078 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:31.078 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:31.078 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:31.078 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:31.078 7f0dd14d4360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:31.079 7f0dd14d4360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:34.018 7f0dd13c8f40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:55:35.079 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:35.080 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:35.080 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:35.080 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:35.080 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:35.080 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:35.080 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:35.081 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:35.081 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:35.081 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:35.081 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:35.081 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:35.081 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:35.081 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:39.084 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:39.086 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:39.086 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:39.086 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:39.086 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:39.086 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:39.086 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:39.086 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:39.087 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:39.087 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:39.087 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:39.087 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:39.087 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:39.087 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:43.087 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:43.089 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:43.089 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:43.089 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:43.089 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:43.089 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:43.089 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:43.090 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:43.090 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:43.090 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:43.090 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:43.090 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:43.090 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:43.090 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:45.997 7f0dd13c8f40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:55:47.090 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:47.091 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:47.092 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:47.092 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:47.092 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:47.092 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:47.092 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:47.092 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:47.092 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:47.093 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:47.093 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:47.093 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:47.093 7f0dd13c8f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:47.093 7f0dd13c8f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:51.096 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:51.097 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:51.098 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:51.098 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:51.098 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:51.098 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:51.098 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:51.099 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:51.099 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:51.099 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:51.099 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:51.099 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:51.099 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:51.100 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:55.102 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:55.103 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:55.103 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:55.103 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:55.103 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:55.103 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:55.104 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:55.104 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:55.104 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:55.104 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:55.104 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:55.104 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:55.105 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:55.105 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:58.064 7f0dd13c8b80 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:55:59.105 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:59.106 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:59.106 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:59.106 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:59.106 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:59.107 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:59.107 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:59.107 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:59.107 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:59.107 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:59.107 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:59.107 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:59.107 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:55:59.107 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:59.430 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 01:55:59.799 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:59.799 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19665792 disk_space_available_1k = 61424692 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 01:55:59.799 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:59.961 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 01:56:00.755 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 01:56:03.107 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:03.109 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:03.109 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:03.109 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:03.109 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:03.109 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:03.109 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:03.110 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:03.110 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:03.110 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:03.110 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:03.110 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:03.110 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:03.110 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:03.792 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19665816, partition_space_available_1k=61424668, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1301028, cpu_share=0.03, mem_res=1293020, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45540, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9331176, free=19899028, buffers=691560, cached=2743356, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.16, five_min_avg=1.75, fifteen_min_avg=1.42, cpu_share=0.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 01:56:03.793 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:03.793 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19665816 partition_space_available_1k = 61424668 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1301028 cpu_share = 0.03 mem_res = 1293020 >>contrail-database-nodemgr : << mem_virt = 45540 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 9331176 free = 19899028 buffers = 691560 cached = 2743356 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.16 five_min_avg = 1.75 fifteen_min_avg = 1.42 cpu_share = 0.88 node_type = database-node >> >> 09/01/2025 01:56:03.793 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:07.110 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:07.111 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:07.111 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:07.112 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:07.112 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:07.112 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:07.112 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:07.112 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:07.112 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:07.112 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:07.113 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:07.113 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:07.113 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:07.113 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:10.142 7f0dd13c8b80 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:56:11.113 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:11.114 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:11.114 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:11.114 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:11.115 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:11.115 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:11.115 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:11.115 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:11.115 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:11.115 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:11.115 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:11.115 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:11.116 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:11.116 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:15.117 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:15.118 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:15.119 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:15.119 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:15.119 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:15.119 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:15.119 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:15.120 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:15.120 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:15.120 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:15.120 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:15.120 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:15.120 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:15.120 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:19.121 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:19.122 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:19.122 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:19.122 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:19.122 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:19.122 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:19.122 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:19.123 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:19.123 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:19.123 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:19.123 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:19.123 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:19.123 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:19.123 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:22.494 7f0dd13c8c20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:56:23.123 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:23.124 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:23.124 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:23.124 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:23.125 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:23.125 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:23.125 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:23.125 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:23.125 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:23.125 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:23.126 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:23.126 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:23.126 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:23.126 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:27.127 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:27.129 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:27.129 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:27.129 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:27.129 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:27.129 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:27.129 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:27.129 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:27.129 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:27.130 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:27.130 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:27.130 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:27.130 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:27.130 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:31.132 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:31.133 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:31.133 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:31.133 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:31.133 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:31.133 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:31.133 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:31.134 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:31.134 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:31.134 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:31.134 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:31.134 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:31.134 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:31.134 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:34.515 7f0dd13c8c20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:56:35.134 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:35.135 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:35.135 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:35.135 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:35.135 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:35.136 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:35.136 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:35.136 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:35.136 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.136 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:35.136 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.136 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:35.136 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:35.137 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:39.139 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:39.140 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:39.141 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:39.141 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:39.141 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:39.141 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:39.141 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:39.141 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:39.141 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:39.142 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:39.142 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:39.142 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:39.142 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:39.142 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:43.145 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:43.146 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:43.146 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:43.147 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:43.147 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:43.147 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:43.147 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:43.147 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:43.147 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:43.147 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:43.147 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:43.147 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:43.148 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:43.148 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:46.497 7f0dd13c8c20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:56:47.148 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:47.149 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:47.149 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:47.149 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:47.149 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:47.149 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:47.149 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:47.150 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:47.150 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:47.150 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:47.150 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:47.150 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:47.150 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:47.150 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:51.153 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:51.154 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:51.154 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:51.154 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:51.154 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:51.154 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:51.155 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:51.155 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:51.155 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:51.155 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:51.155 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:51.155 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:51.155 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:51.155 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:55.159 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:55.160 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:55.160 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:55.160 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:55.160 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:55.160 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:55.161 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:55.161 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:55.161 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:55.161 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:55.161 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:55.161 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:55.161 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:55.161 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:58.555 7f0dd13c8b80 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:56:59.161 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:59.162 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:59.162 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:59.162 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:59.162 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:59.162 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:59.162 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:59.163 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:59.163 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:59.163 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:59.163 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:59.163 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:59.163 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:56:59.163 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:59.427 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 01:56:59.758 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:59.758 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19669216 disk_space_available_1k = 61421268 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 01:56:59.758 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:59.921 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 01:57:00.711 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 01:57:03.163 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:03.164 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:03.164 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:03.164 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:03.165 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:03.165 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:03.165 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:03.165 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:03.165 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:03.165 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:03.165 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:03.165 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:03.166 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:03.166 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:03.739 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19669256, partition_space_available_1k=61421228, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1301720, cpu_share=0.04, mem_res=1293400, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45636, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9369252, free=19851220, buffers=691620, cached=2753028, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.89, five_min_avg=1.59, fifteen_min_avg=1.38, cpu_share=0.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 01:57:03.740 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:03.740 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19669256 partition_space_available_1k = 61421228 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1301720 cpu_share = 0.04 mem_res = 1293400 >>contrail-database-nodemgr : << mem_virt = 45636 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 9369252 free = 19851220 buffers = 691620 cached = 2753028 node_type = database-node >> system_cpu_usage = << one_min_avg = 0.89 five_min_avg = 1.59 fifteen_min_avg = 1.38 cpu_share = 0.88 node_type = database-node >> >> 09/01/2025 01:57:03.740 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:07.168 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:07.169 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:07.169 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:07.169 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:07.169 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:07.169 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:07.169 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:07.170 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:07.170 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:07.170 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:07.170 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:07.170 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:07.170 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:07.170 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:10.597 7f0dd168d360 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:57:11.170 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:11.171 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:11.172 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:11.172 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:11.172 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:11.172 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:11.172 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:11.173 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:11.173 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:11.173 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:11.173 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:11.173 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:11.174 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:11.174 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:15.174 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:15.176 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:15.176 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:15.176 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:15.176 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:15.176 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:15.176 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:15.176 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:15.177 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:15.177 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:15.177 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:15.177 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:15.177 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:15.177 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:19.177 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:19.178 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:19.178 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:19.178 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:19.178 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:19.178 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:19.178 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:19.179 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:19.179 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:19.179 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:19.179 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:19.179 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:19.179 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:19.179 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:22.947 7f0dd14d4f40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:57:23.179 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:23.180 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:23.180 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:23.180 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:23.180 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:23.180 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:23.180 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:23.181 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:23.181 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:23.181 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:23.181 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:23.181 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:23.181 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:23.181 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:27.184 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:27.185 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:27.185 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:27.185 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:27.186 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:27.186 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:27.186 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:27.186 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:27.186 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:27.186 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:27.186 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:27.186 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:27.187 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:27.187 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:31.190 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:31.192 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:31.192 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:31.192 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:31.192 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:31.192 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:31.193 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:31.193 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:31.193 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:31.193 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:31.193 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:31.193 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:31.193 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:31.193 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:35.041 7f0dd168d360 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:57:35.192 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:35.193 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:35.193 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:35.194 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:35.194 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:35.194 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:35.194 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:35.194 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:35.194 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:35.195 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:35.195 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:35.195 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:35.195 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:35.195 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:39.198 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:39.199 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:39.200 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:39.200 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:39.200 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:39.200 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:39.200 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:39.200 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:39.200 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:39.200 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:39.201 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:39.201 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:39.201 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:39.201 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:43.204 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:43.205 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:43.205 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:43.205 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:43.206 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:43.206 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:43.206 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:43.206 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:43.206 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:43.206 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:43.206 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:43.206 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:43.206 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:43.206 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:47.098 7f0dd168d360 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:57:47.206 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:47.207 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:47.207 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:47.207 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:47.207 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:47.207 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:47.207 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:47.208 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:47.208 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:47.208 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:47.208 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:47.208 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:47.208 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:47.208 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:51.208 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:51.209 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:51.209 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:51.209 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:51.209 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:51.209 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:51.209 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:51.210 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:51.210 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:51.210 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:51.210 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:51.210 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:51.210 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:51.210 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:55.212 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:55.213 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:55.213 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:55.213 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:55.213 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:55.213 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:55.214 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:55.214 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:55.214 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:55.214 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:55.214 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:55.214 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:55.214 7f0dd168d360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:55.214 7f0dd168d360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:59.045 7f0dd168d360 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:57:59.216 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:59.217 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:59.217 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:59.217 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:59.217 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:59.217 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:59.217 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:59.218 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:59.218 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:59.218 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:59.218 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:59.218 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:59.218 7f0dd14d4400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:57:59.218 7f0dd14d4400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:59.477 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 01:57:59.814 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:59.815 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19672488 disk_space_available_1k = 61417996 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 01:57:59.815 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:59.978 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 01:58:00.743 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 01:58:03.218 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:03.220 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:03.220 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:03.220 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:03.220 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:03.220 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:03.220 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:03.221 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:03.221 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:03.221 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:03.221 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:03.221 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:03.221 7f0dd14c3ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:03.221 7f0dd14c3ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:03.771 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19672524, partition_space_available_1k=61417960, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1301824, cpu_share=0.03, mem_res=1293476, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45732, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9340304, free=19882544, buffers=691708, cached=2750564, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.72, five_min_avg=1.42, fifteen_min_avg=1.34, cpu_share=0.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 01:58:03.771 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:03.772 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19672524 partition_space_available_1k = 61417960 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1301824 cpu_share = 0.03 mem_res = 1293476 >>contrail-database-nodemgr : << mem_virt = 45732 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 9340304 free = 19882544 buffers = 691708 cached = 2750564 node_type = database-node >> system_cpu_usage = << one_min_avg = 0.72 five_min_avg = 1.42 fifteen_min_avg = 1.34 cpu_share = 0.88 node_type = database-node >> >> 09/01/2025 01:58:03.772 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:07.223 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:07.225 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:07.225 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:07.225 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:07.225 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:07.225 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:07.225 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:07.225 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:07.226 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:07.226 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:07.226 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:07.226 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:07.226 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:07.226 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:11.139 7f0dd14c3a40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:58:11.226 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:11.227 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:11.227 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:11.227 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:11.227 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:11.227 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:11.227 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:11.227 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:11.228 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:11.228 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:11.228 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:11.228 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:11.228 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:11.228 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:15.230 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:15.232 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:15.232 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:15.232 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:15.232 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:15.232 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:15.232 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:15.233 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:15.233 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:15.233 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:15.233 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:15.233 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:15.233 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:15.233 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:19.233 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:19.234 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:19.234 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:19.234 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:19.234 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:19.235 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:19.235 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:19.235 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:19.235 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:19.235 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:19.235 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:19.235 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:19.235 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:19.236 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:23.238 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:23.239 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:23.239 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:23.239 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:23.240 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:23.240 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:23.240 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:23.240 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:23.240 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:23.241 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:23.241 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:23.241 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:23.241 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:23.241 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:23.500 7f0dd14d4a40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:58:27.240 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:27.241 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:27.241 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:27.241 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:27.241 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:27.241 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:27.241 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:27.242 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:27.242 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:27.242 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:27.242 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:27.242 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:27.242 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:27.242 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:31.245 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:31.247 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:31.247 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:31.247 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:31.247 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:31.247 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:31.247 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:31.247 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:31.248 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:31.248 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:31.248 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:31.248 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:31.248 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:31.248 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:35.248 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:35.249 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:35.249 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:35.249 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:35.250 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:35.250 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:35.250 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:35.250 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:35.250 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:35.250 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:35.250 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:35.250 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:35.251 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:35.251 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:35.510 7f0dd14c34a0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:58:39.253 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:39.254 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:39.254 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:39.254 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:39.254 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:39.254 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:39.255 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:39.255 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:39.255 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:39.255 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:39.255 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:39.255 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:39.255 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:39.255 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:43.259 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:43.260 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:43.260 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:43.260 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:43.260 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:43.260 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:43.260 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:43.261 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:43.261 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:43.261 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:43.261 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:43.261 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:43.261 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:43.261 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:47.264 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:47.265 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:47.265 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:47.265 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:47.265 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:47.265 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:47.265 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:47.266 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:47.266 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:47.266 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:47.266 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:47.266 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:47.266 7f0dd14c34a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:47.266 7f0dd14c34a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:47.518 7f0dd14c34a0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:58:51.268 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:51.269 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:51.269 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:51.269 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:51.269 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:51.269 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:51.269 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:51.270 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:51.270 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:51.270 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:51.270 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:51.270 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:51.270 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:51.270 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:55.274 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:55.275 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:55.275 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:55.275 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:55.275 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:55.275 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:55.275 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:55.276 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:55.276 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:55.276 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:55.276 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:55.276 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:55.276 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:55.276 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:59.276 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:59.279 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:59.280 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:59.280 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:59.280 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:59.280 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:59.280 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:59.281 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:59.281 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:59.281 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:59.281 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:59.281 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:59.281 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:58:59.281 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:59.458 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 01:58:59.585 7f0dd14d44a0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:58:59.816 7f0dd14d44a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:59.816 7f0dd14d44a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19680836 disk_space_available_1k = 61409648 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 01:58:59.816 7f0dd14d44a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:59.989 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 01:59:00.767 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 01:59:03.282 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:03.283 7f0dd14c3220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:03.284 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:03.284 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:03.284 7f0dd14c3220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:03.284 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:03.284 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:03.284 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:03.284 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:03.285 7f0dd14c3220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:03.285 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:03.285 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:03.285 7f0dd14c3220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:03.285 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:03.796 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19676320, partition_space_available_1k=61414164, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1303256, cpu_share=0.03, mem_res=1294868, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45824, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9356796, free=19861420, buffers=691944, cached=2754960, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.81, five_min_avg=1.33, fifteen_min_avg=1.31, cpu_share=0.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 01:59:03.796 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:03.797 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19676320 partition_space_available_1k = 61414164 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1303256 cpu_share = 0.03 mem_res = 1294868 >>contrail-database-nodemgr : << mem_virt = 45824 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 9356796 free = 19861420 buffers = 691944 cached = 2754960 node_type = database-node >> system_cpu_usage = << one_min_avg = 0.81 five_min_avg = 1.33 fifteen_min_avg = 1.31 cpu_share = 0.88 node_type = database-node >> >> 09/01/2025 01:59:03.797 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:07.286 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:07.287 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:07.287 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:07.287 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:07.288 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:07.288 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:07.288 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:07.288 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:07.288 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:07.289 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:07.289 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:07.289 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:07.289 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:07.289 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:11.291 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:11.292 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:11.292 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:11.293 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:11.293 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:11.293 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:11.293 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:11.293 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:11.293 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:11.293 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:11.293 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:11.293 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:11.294 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:11.294 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:11.579 7f0dd1659d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:59:15.297 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:15.298 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:15.298 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:15.298 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:15.298 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:15.298 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:15.298 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:15.299 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:15.299 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:15.299 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:15.299 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:15.299 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:15.299 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:15.299 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:19.299 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:19.299 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:19.300 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:19.300 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:19.300 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:19.300 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:19.300 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:19.300 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:19.300 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:19.301 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:19.301 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:19.301 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:19.301 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:19.301 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:23.304 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:23.305 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:23.305 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:23.305 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:23.305 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:23.305 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:23.305 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:23.306 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:23.306 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:23.306 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:23.306 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:23.306 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:23.306 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:23.306 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:23.923 7f0dd14d4ae0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:59:27.308 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:27.309 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:27.310 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:27.310 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:27.310 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:27.310 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:27.310 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:27.311 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:27.311 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:27.311 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:27.311 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:27.311 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:27.311 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:27.312 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:31.312 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:31.313 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:31.313 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:31.313 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:31.313 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:31.313 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:31.313 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:31.314 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:31.314 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:31.314 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:31.314 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:31.314 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:31.314 7f0dd14d4ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:31.314 7f0dd14d4ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:35.314 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:35.315 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:35.315 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:35.315 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:35.316 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:35.316 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:35.316 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:35.316 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:35.316 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:35.316 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:35.316 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:35.317 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:35.317 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:35.317 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:35.986 7f0dd14d4d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:59:39.318 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:39.320 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:39.320 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:39.320 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:39.320 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:39.320 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:39.320 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:39.320 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:39.322 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.322 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:39.322 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.322 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.323 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:39.323 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:43.324 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:43.325 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:43.325 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:43.325 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:43.325 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:43.325 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:43.325 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:43.326 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:43.326 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:43.326 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:43.326 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:43.326 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:43.326 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:43.326 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:47.328 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:47.329 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:47.330 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:47.330 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:47.330 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:47.330 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:47.330 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:47.330 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:47.331 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:47.331 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:47.331 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:47.331 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:47.331 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:47.331 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:48.021 7f0dd14d4d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:59:51.332 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:51.333 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:51.333 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:51.333 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:51.333 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:51.334 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:51.334 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:51.334 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:51.334 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:51.334 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:51.334 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:51.334 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:51.334 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:51.335 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:55.336 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:55.337 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:55.337 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:55.337 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:55.337 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:55.337 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:55.337 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:55.338 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:55.338 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:55.338 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:55.338 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:55.338 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:55.338 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:55.338 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:59.337 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:59.338 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:59.338 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:59.338 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:59.338 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:59.339 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:59.339 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:59.339 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:59.339 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:59.339 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:59.339 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:59.339 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:59.339 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:59.339 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:59.438 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 01:59:59.778 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:59.778 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19682636 disk_space_available_1k = 61407848 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 01:59:59.778 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:59.962 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:00:00.174 7f0dd13c8ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:00:00.805 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:00:03.340 7f0dd14c3d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:03.342 7f0dd14c3d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:03.342 7f0dd14c3d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:03.342 7f0dd14c3d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:03.343 7f0dd14c3d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:03.343 7f0dd14c3d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:03.343 7f0dd14c3d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:03.343 7f0dd14c3d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:03.343 7f0dd14c3d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:03.344 7f0dd14c3d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:03.344 7f0dd14c3d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:03.344 7f0dd14c3d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:03.344 7f0dd14c3d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:03.344 7f0dd14c3d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:03.833 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19680204, partition_space_available_1k=61410280, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1303364, cpu_share=0.03, mem_res=1294944, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45912, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9354360, free=19859552, buffers=692000, cached=2759208, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.48, five_min_avg=1.14, fifteen_min_avg=1.25, cpu_share=0.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:00:03.833 7f0dd14d44a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:03.834 7f0dd14d44a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19680204 partition_space_available_1k = 61410280 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1303364 cpu_share = 0.03 mem_res = 1294944 >>contrail-database-nodemgr : << mem_virt = 45912 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 9354360 free = 19859552 buffers = 692000 cached = 2759208 node_type = database-node >> system_cpu_usage = << one_min_avg = 0.48 five_min_avg = 1.14 fifteen_min_avg = 1.25 cpu_share = 0.88 node_type = database-node >> >> 09/01/2025 02:00:03.834 7f0dd14d44a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:07.345 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:07.346 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:07.346 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:07.346 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:07.347 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:07.347 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:07.347 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:07.347 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:07.347 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:07.347 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:07.347 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:07.347 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:07.348 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:07.348 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:11.350 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:11.351 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:11.351 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:11.351 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:11.351 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:11.352 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:11.352 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:11.352 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:11.352 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:11.352 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:11.352 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:11.352 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:11.352 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:11.352 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:12.308 7f0dd14d4720 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:00:15.353 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:15.355 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:15.355 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:15.355 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:15.355 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:15.355 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:15.355 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:15.356 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:15.356 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:15.356 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:15.356 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:15.356 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:15.356 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:15.356 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:19.356 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:19.357 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:19.357 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:19.357 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:19.358 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:19.358 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:19.358 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:19.358 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:19.358 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:19.358 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:19.358 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:19.358 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:19.359 7f0dd14d4720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:19.359 7f0dd14d4720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:23.360 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:23.361 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:23.361 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:23.361 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:23.361 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:23.361 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:23.361 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:23.362 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:23.362 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:23.362 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:23.362 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:23.362 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:23.362 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:23.362 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:24.788 7f0dd13c8900 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:00:27.365 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:27.366 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:27.366 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:27.366 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:27.366 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:27.366 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:27.366 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:27.366 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:27.367 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:27.367 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:27.367 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:27.367 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:27.367 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:27.367 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:31.370 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:31.371 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:31.372 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:31.372 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:31.372 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:31.372 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:31.372 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:31.372 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:31.372 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:31.373 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:31.373 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:31.373 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:31.373 7f0dd13c8900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:31.373 7f0dd13c8900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:35.373 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:35.374 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:35.374 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:35.374 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:35.374 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:35.375 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:35.375 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:35.375 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:35.375 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:35.375 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:35.375 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:35.376 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:35.376 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:35.376 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:36.903 7f0dd14c37c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:00:39.377 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:39.378 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:39.378 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:39.378 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:39.378 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:39.379 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:39.379 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:39.379 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:39.379 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:39.379 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:39.379 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:39.379 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:39.379 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:39.379 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:43.380 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:43.381 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:43.381 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:43.381 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:43.381 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:43.381 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:43.381 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:43.382 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:43.382 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:43.382 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:43.382 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:43.382 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:43.382 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:43.382 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:47.384 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:47.385 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:47.385 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:47.385 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:47.385 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:47.385 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:47.385 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:47.386 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:47.386 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:47.386 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:47.386 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:47.386 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:47.386 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:47.386 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:48.958 7f0dd14c37c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:00:51.387 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:51.388 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:51.388 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:51.388 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:51.388 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:51.389 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:51.389 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:51.389 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:51.389 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:51.389 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:51.389 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:51.389 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:51.389 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:51.389 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:55.392 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:55.393 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:55.393 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:55.393 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:55.393 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:55.393 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:55.393 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:55.394 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:55.394 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:55.394 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:55.394 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:55.394 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:55.394 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:55.394 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:59.394 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:59.395 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:59.395 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:59.395 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:59.395 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:59.395 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:59.396 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:59.396 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:59.396 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:59.396 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:59.396 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:59.396 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:59.396 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:59.396 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:59.474 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:00:59.806 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:59.806 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19682164 disk_space_available_1k = 61408320 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:00:59.806 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:59.978 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:01:00.759 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:01:01.144 7f0dd14d44a0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:01:03.397 7f0dd14d44a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:03.398 7f0dd14d44a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:03.399 7f0dd14d44a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:03.399 7f0dd14d44a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:03.399 7f0dd14d44a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:03.399 7f0dd14d44a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:03.399 7f0dd14d44a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:03.399 7f0dd14d44a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:03.399 7f0dd14d44a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:03.400 7f0dd14d44a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:03.400 7f0dd14d44a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:03.400 7f0dd14d44a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:03.400 7f0dd14d44a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:03.400 7f0dd14d44a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:03.796 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19687140, partition_space_available_1k=61403344, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1303468, cpu_share=0.03, mem_res=1295020, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46016, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9377016, free=19832892, buffers=692064, cached=2763148, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.48, five_min_avg=1.03, fifteen_min_avg=1.2, cpu_share=0.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:01:03.796 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:03.796 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19687140 partition_space_available_1k = 61403344 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1303468 cpu_share = 0.03 mem_res = 1295020 >>contrail-database-nodemgr : << mem_virt = 46016 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 9377016 free = 19832892 buffers = 692064 cached = 2763148 node_type = database-node >> system_cpu_usage = << one_min_avg = 0.48 five_min_avg = 1.03 fifteen_min_avg = 1.2 cpu_share = 0.88 node_type = database-node >> >> 09/01/2025 02:01:03.796 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:07.401 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:07.402 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:07.402 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:07.402 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:07.403 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:07.403 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:07.403 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:07.403 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:07.403 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:07.403 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:07.403 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:07.403 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:07.403 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:07.403 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:11.407 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:11.408 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:11.408 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:11.408 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:11.408 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:11.408 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:11.409 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:11.409 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:11.409 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:11.409 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:11.409 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:11.409 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:11.409 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:11.409 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:13.289 7f0dd14c3720 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:01:15.411 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:15.412 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:15.412 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:15.412 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:15.412 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:15.413 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:15.413 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:15.413 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:15.413 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:15.413 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:15.413 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:15.413 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:15.413 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:15.413 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:19.417 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:19.418 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:19.418 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:19.418 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:19.418 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:19.418 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:19.418 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:19.419 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:19.419 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:19.419 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:19.419 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:19.419 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:19.419 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:19.419 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:23.421 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:23.423 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:23.423 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:23.423 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:23.423 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:23.423 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:23.423 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:23.423 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:23.424 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:23.424 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:23.424 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:23.424 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:23.424 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:23.424 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:25.722 7f0dd14d4d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:01:27.424 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:27.425 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:27.425 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:27.425 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:27.425 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:27.425 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:27.425 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:27.426 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:27.426 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:27.426 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:27.426 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:27.426 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:27.426 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:27.426 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:31.428 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:31.429 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:31.429 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:31.429 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:31.429 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:31.429 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:31.429 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:31.430 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:31.430 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:31.430 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:31.430 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:31.430 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:31.430 7f0dd14d4d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:31.430 7f0dd14d4d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:35.430 7f0dd14c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:35.432 7f0dd14c3c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:35.432 7f0dd14c3c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:35.432 7f0dd14c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:35.432 7f0dd14c3c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:35.432 7f0dd14c3c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:35.433 7f0dd14c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:35.433 7f0dd14c3c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:35.433 7f0dd14c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:35.433 7f0dd14c3c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:35.433 7f0dd14c3c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:35.433 7f0dd14c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:35.433 7f0dd14c3c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:35.433 7f0dd14c3c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:37.850 7f0dd14c30e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:01:39.435 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:39.436 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:39.436 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:39.436 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:39.436 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:39.436 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:39.436 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:39.437 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:39.437 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:39.437 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:39.437 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:39.437 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:39.437 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:39.437 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:43.440 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:43.441 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:43.442 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:43.442 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:43.442 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:43.442 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:43.442 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:43.442 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:43.442 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:43.442 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:43.442 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:43.443 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:43.443 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:43.443 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:47.446 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:47.447 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:47.447 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:47.447 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:47.448 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:47.448 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:47.448 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:47.448 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:47.448 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:47.448 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:47.448 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:47.448 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:47.449 7f0dd14c30e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:47.449 7f0dd14c30e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:49.959 7f0dd14c30e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:01:51.448 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:51.449 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:51.449 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:51.449 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:51.449 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:51.450 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:51.450 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:51.450 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:51.450 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:51.450 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:51.450 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:51.450 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:51.450 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:51.450 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:55.454 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:55.455 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:55.455 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:55.455 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:55.455 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:55.455 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:55.455 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:55.456 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:55.456 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:55.456 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:55.456 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:55.456 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:55.456 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:55.456 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:59.452 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:01:59.455 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:59.457 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:59.457 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:59.457 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:59.457 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:59.457 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:59.457 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:59.457 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:59.458 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:59.458 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:59.458 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:59.458 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:59.458 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:59.458 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:59.782 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:59.783 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19685712 disk_space_available_1k = 61404772 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:01:59.783 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:59.962 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:02:00.825 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:02:02.118 7f0dd14c3220 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:02:03.458 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:03.460 7f0dd14c3220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:03.460 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:03.460 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:03.460 7f0dd14c3220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:03.460 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:03.460 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:03.461 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:03.461 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:03.461 7f0dd14c3220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:03.461 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:03.461 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:03.461 7f0dd14c3220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:03.461 7f0dd14c3220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:03.857 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19687212, partition_space_available_1k=61403272, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1303756, cpu_share=0.03, mem_res=1295096, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46104, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9362244, free=19843680, buffers=692116, cached=2767080, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.42, five_min_avg=0.92, fifteen_min_avg=1.15, cpu_share=0.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:02:03.857 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:03.857 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19687212 partition_space_available_1k = 61403272 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1303756 cpu_share = 0.03 mem_res = 1295096 >>contrail-database-nodemgr : << mem_virt = 46104 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 9362244 free = 19843680 buffers = 692116 cached = 2767080 node_type = database-node >> system_cpu_usage = << one_min_avg = 0.42 five_min_avg = 0.92 fifteen_min_avg = 1.15 cpu_share = 0.88 node_type = database-node >> >> 09/01/2025 02:02:03.857 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:07.462 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:07.463 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:07.463 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:07.463 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:07.463 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:07.463 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:07.463 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:07.464 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:07.464 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:07.464 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:07.464 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:07.464 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:07.464 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:07.464 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:11.467 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:11.468 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:11.468 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:11.468 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:11.468 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:11.468 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:11.469 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:11.469 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:11.469 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:11.469 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:11.469 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:11.469 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:11.469 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:11.469 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:14.137 7f0dd13c8b80 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:02:15.470 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:15.471 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:15.471 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:15.471 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:15.471 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:15.471 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:15.471 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:15.472 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:15.472 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:15.472 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:15.472 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:15.472 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:15.472 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:15.473 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:19.472 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:19.473 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:19.474 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:19.474 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:19.474 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:19.474 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:19.474 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:19.474 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:19.474 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:19.474 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:19.475 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:19.475 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:19.475 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:19.475 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:23.475 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:23.476 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:23.476 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:23.476 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:23.477 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:23.477 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:23.477 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:23.477 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:23.477 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:23.477 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:23.477 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:23.477 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:23.478 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:23.478 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:26.613 7f0dd1659d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:02:27.477 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:27.478 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:27.479 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:27.479 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:27.479 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:27.479 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:27.479 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:27.479 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:27.479 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:27.479 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:27.480 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:27.480 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:27.480 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:27.480 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:31.482 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:31.483 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:31.483 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:31.483 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:31.483 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:31.483 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:31.483 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:31.484 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:31.484 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:31.484 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:31.484 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:31.484 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:31.484 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:31.484 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:35.486 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:35.487 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:35.487 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:35.487 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:35.487 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:35.487 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:35.488 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:35.488 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:35.488 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:35.488 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:35.488 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:35.488 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:35.488 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:35.489 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:38.726 7f0dd13c8c20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:02:39.488 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:39.489 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:39.489 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:39.489 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:39.489 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:39.489 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:39.490 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:39.490 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:39.490 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:39.490 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:39.490 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:39.490 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:39.490 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:39.490 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:43.493 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:43.494 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:43.494 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:43.494 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:43.494 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:43.495 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:43.495 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:43.495 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:43.495 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:43.495 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:43.495 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:43.495 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:43.495 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:43.495 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:47.499 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:47.500 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:47.500 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:47.500 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:47.500 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:47.501 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:47.501 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:47.501 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:47.501 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:47.501 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:47.502 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:47.502 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:47.502 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:47.502 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:50.746 7f0dd13c8c20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:02:51.501 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:51.502 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:51.502 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:51.502 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:51.502 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:51.502 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:51.502 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:51.503 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:51.503 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:51.503 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:51.503 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:51.503 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:51.503 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:51.503 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:55.505 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:55.506 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:55.506 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:55.506 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:55.506 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:55.506 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:55.506 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:55.507 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:55.507 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:55.507 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:55.507 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:55.507 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:55.507 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:55.507 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:59.450 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:02:59.507 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:59.509 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:59.509 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:59.509 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:59.509 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:59.510 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:59.510 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:59.510 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:59.510 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:59.510 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:59.510 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:59.510 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:59.510 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:59.510 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:59.783 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:59.783 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19689040 disk_space_available_1k = 61401444 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:02:59.783 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:59.946 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:03:00.715 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:03:02.778 7f0dd14c3900 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:03:03.511 7f0dd14c3900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:03.512 7f0dd14c3900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:03.512 7f0dd14c3900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:03.512 7f0dd14c3900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:03.513 7f0dd14c3900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:03.513 7f0dd14c3900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:03.513 7f0dd14c3900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:03.513 7f0dd14c3900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:03.513 7f0dd14c3900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:03.513 7f0dd14c3900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:03.513 7f0dd14c3900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:03.513 7f0dd14c3900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:03.514 7f0dd14c3900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:03.514 7f0dd14c3900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:03.744 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19690440, partition_space_available_1k=61400044, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1303864, cpu_share=0.03, mem_res=1295172, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46316, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9391432, free=19811152, buffers=692152, cached=2770384, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.5, five_min_avg=0.84, fifteen_min_avg=1.11, cpu_share=0.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:03:03.744 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:03.744 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19690440 partition_space_available_1k = 61400044 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1303864 cpu_share = 0.03 mem_res = 1295172 >>contrail-database-nodemgr : << mem_virt = 46316 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 9391432 free = 19811152 buffers = 692152 cached = 2770384 node_type = database-node >> system_cpu_usage = << one_min_avg = 0.5 five_min_avg = 0.84 fifteen_min_avg = 1.11 cpu_share = 0.88 node_type = database-node >> >> 09/01/2025 02:03:03.745 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:07.514 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:07.515 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:07.515 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:07.516 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:07.516 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:07.516 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:07.516 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:07.516 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:07.516 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:07.516 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:07.516 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:07.517 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:07.517 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:07.517 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:11.520 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:11.521 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:11.521 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:11.521 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:11.521 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:11.521 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:11.522 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:11.522 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:11.522 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:11.522 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:11.522 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:11.522 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:11.522 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:11.522 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:14.898 7f0dd13c8ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:03:15.522 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:15.523 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:15.524 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:15.524 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:15.524 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:15.524 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:15.524 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:15.524 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:15.524 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:15.525 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:15.525 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:15.525 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:15.525 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:15.525 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:19.528 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:19.529 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:19.529 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:19.529 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:19.529 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:19.529 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:19.529 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:19.530 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:19.530 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:19.530 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:19.530 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:19.530 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:19.530 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:19.530 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:23.530 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:23.531 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:23.531 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:23.531 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:23.531 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:23.531 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:23.531 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:23.532 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:23.532 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:23.532 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:23.532 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:23.532 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:23.532 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:23.532 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:27.412 7f0dd14c3040 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:03:27.531 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:27.532 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:27.532 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:27.533 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:27.533 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:27.533 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:27.533 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:27.533 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:27.533 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:27.533 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:27.534 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:27.534 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:27.534 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:27.534 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:31.537 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:31.538 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:31.538 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:31.538 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:31.539 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:31.539 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:31.539 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:31.539 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:31.539 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:31.539 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:31.539 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:31.539 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:31.539 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:31.540 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:35.541 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:35.542 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:35.542 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:35.542 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:35.542 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:35.542 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:35.542 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:35.543 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:35.543 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:35.543 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:35.543 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:35.543 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:35.543 7f0dd14c3040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:35.543 7f0dd14c3040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:39.544 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:39.545 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:39.546 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:39.546 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:39.546 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:39.546 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:39.546 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:39.546 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:39.546 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:39.546 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:39.547 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:39.547 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:39.547 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:39.547 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:39.561 7f0dd13c8ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:03:43.550 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:43.551 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:43.551 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:43.551 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:43.551 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:43.551 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:43.551 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:43.552 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:43.552 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:43.552 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:43.552 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:43.552 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:43.552 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:43.552 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:47.555 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:47.556 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:47.556 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:47.556 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:47.557 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:47.557 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:47.557 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:47.557 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:47.557 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:47.557 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:47.557 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:47.557 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:47.557 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:47.558 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:51.560 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:51.561 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:51.561 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:51.561 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:51.561 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:51.561 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:51.561 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:51.562 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:51.562 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:51.562 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:51.562 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:51.562 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:51.562 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:51.562 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:51.658 7f0dd14d45e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:03:55.564 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:55.565 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:55.565 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:55.565 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:55.565 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:55.566 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:55.566 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:55.566 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:55.566 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:55.566 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:55.566 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:55.566 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:55.566 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:55.566 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:59.449 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:03:59.565 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:59.568 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:59.568 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:59.568 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:59.568 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:59.568 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:59.569 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:59.569 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:59.569 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:59.569 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:59.569 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:59.569 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:59.569 7f0dd14d4a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:59.569 7f0dd14d4a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:59.787 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:59.787 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19692392 disk_space_available_1k = 61398092 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:03:59.787 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:59.954 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:04:00.729 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:04:03.570 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:03.571 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:03.571 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:03.571 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:03.571 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:03.572 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:03.572 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:03.572 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:03.572 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:03.572 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:03.572 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:03.572 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:03.572 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:03.573 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:03.758 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19692432, partition_space_available_1k=61398052, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1303972, cpu_share=0.03, mem_res=1295248, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46280, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9441388, free=19753588, buffers=692212, cached=2777932, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.34, five_min_avg=0.73, fifteen_min_avg=1.05, cpu_share=0.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:04:03.759 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:03.759 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19692432 partition_space_available_1k = 61398052 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1303972 cpu_share = 0.03 mem_res = 1295248 >>contrail-database-nodemgr : << mem_virt = 46280 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 9441388 free = 19753588 buffers = 692212 cached = 2777932 node_type = database-node >> system_cpu_usage = << one_min_avg = 0.34 five_min_avg = 0.73 fifteen_min_avg = 1.05 cpu_share = 0.88 node_type = database-node >> >> 09/01/2025 02:04:03.760 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:03.793 7f0dd13c8c20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:04:07.573 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:07.574 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:07.574 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:07.574 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:07.574 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:07.574 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:07.574 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:07.575 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:07.575 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:07.575 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:07.575 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:07.575 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:07.575 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:07.575 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:11.576 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:11.577 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:11.577 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:11.577 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:11.577 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:11.577 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:11.577 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:11.578 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:11.578 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:11.578 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:11.578 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:11.578 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:11.578 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:11.578 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:15.580 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:15.581 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:15.581 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:15.581 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:15.581 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:15.581 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:15.581 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:15.582 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:15.582 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:15.582 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:15.582 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:15.582 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:15.582 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:15.582 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:15.885 7f0dd14d4f40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:04:19.584 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:19.585 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:19.586 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:19.586 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:19.586 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:19.586 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:19.586 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:19.587 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:19.587 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:19.587 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:19.587 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:19.587 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:19.587 7f0dd14d4f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:19.587 7f0dd14d4f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:23.587 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:23.588 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:23.588 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:23.588 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:23.588 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:23.589 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:23.589 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:23.589 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:23.589 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:23.589 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:23.589 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:23.589 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:23.589 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:23.589 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:27.592 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:27.593 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:27.593 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:27.593 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:27.593 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:27.593 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:27.593 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:27.594 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:27.594 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:27.594 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:27.594 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:27.594 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:27.594 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:27.594 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:28.331 7f0dd13c8ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:04:31.597 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:31.598 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:31.598 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:31.598 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:31.598 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:31.598 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:31.599 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:31.599 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:31.599 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:31.599 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:31.599 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:31.599 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:31.599 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:31.599 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:35.601 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:35.602 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:35.603 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:35.603 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:35.603 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:35.603 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:35.603 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:35.603 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:35.603 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:35.603 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:35.603 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:35.604 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:35.604 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:35.604 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:39.606 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:39.607 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:39.607 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:39.607 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:39.607 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:39.607 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:39.607 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:39.608 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:39.608 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:39.608 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:39.608 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:39.608 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:39.608 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:39.608 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:40.466 7f0dd14d4cc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:04:43.608 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:43.609 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:43.609 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:43.609 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:43.609 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:43.609 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:43.609 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:43.610 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:43.610 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:43.610 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:43.610 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:43.610 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:43.610 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:43.610 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:47.611 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:47.612 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:47.612 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:47.612 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:47.612 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:47.612 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:47.612 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:47.613 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:47.613 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:47.613 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:47.613 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:47.613 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:47.613 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:47.613 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:51.616 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:51.617 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:51.617 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:51.617 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:51.617 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:51.617 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:51.617 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:51.618 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:51.618 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:51.618 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:51.618 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:51.618 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:51.618 7f0dd14d4cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:51.618 7f0dd14d4cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:52.589 7f0dd14d4900 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:04:55.620 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:55.621 7f0dd14d4900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:55.621 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:55.621 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:55.621 7f0dd14d4900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:55.621 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:55.622 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:55.622 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:55.622 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:55.622 7f0dd14d4900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:55.622 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:55.622 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:55.622 7f0dd14d4900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:55.622 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.456 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:04:59.621 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:59.624 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:59.624 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:59.624 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:59.624 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:59.624 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:59.624 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:59.625 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:59.625 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.625 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:59.625 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.625 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.625 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:59.625 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.794 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.795 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19696280 disk_space_available_1k = 61394204 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:04:59.795 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.960 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:05:00.730 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:05:03.627 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:03.628 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:03.628 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:03.628 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:03.629 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:03.629 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:03.629 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:03.629 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:03.629 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:03.629 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:03.629 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:03.630 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:03.630 7f0dd14c3a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:03.630 7f0dd14c3a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:03.757 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19696312, partition_space_available_1k=61394172, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1304100, cpu_share=0.03, mem_res=1295324, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46380, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9396076, free=19797900, buffers=692268, cached=2778876, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.62, five_min_avg=0.75, fifteen_min_avg=1.03, cpu_share=0.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:05:03.758 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:03.758 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19696312 partition_space_available_1k = 61394172 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1304100 cpu_share = 0.03 mem_res = 1295324 >>contrail-database-nodemgr : << mem_virt = 46380 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 9396076 free = 19797900 buffers = 692268 cached = 2778876 node_type = database-node >> system_cpu_usage = << one_min_avg = 0.62 five_min_avg = 0.75 fifteen_min_avg = 1.03 cpu_share = 0.88 node_type = database-node >> >> 09/01/2025 02:05:03.758 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:04.636 7f0dd13c8ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:07.629 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:07.630 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:07.630 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:07.630 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:07.630 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:07.630 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:07.630 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:07.631 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:07.631 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:07.631 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:07.631 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:07.631 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:07.631 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:07.631 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:11.634 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:11.635 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:11.636 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:11.636 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:11.636 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:11.636 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:11.636 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:11.636 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:11.637 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:11.637 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:11.637 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:11.637 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:11.637 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:11.637 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:15.640 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:15.641 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:15.641 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:15.642 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:15.642 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:15.642 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:15.642 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:15.642 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:15.642 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:15.642 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:15.642 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:15.643 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:15.643 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:15.643 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:16.742 7f0dd14d4e00 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:19.645 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:19.646 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:19.646 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:19.647 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:19.647 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:19.647 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:19.647 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:19.647 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:19.647 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:19.647 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:19.647 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:19.648 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:19.648 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:19.648 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:23.648 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:23.649 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:23.649 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:23.650 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:23.650 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:23.650 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:23.650 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:23.650 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:23.650 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:23.650 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:23.650 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:23.650 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:23.651 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:23.651 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:27.652 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:27.653 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:27.653 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:27.653 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:27.653 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:27.653 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:27.653 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:27.654 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:27.654 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:27.654 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:27.654 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:27.654 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:27.654 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:27.654 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:29.164 7f0dd14d4e00 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:31.656 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:31.657 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:31.657 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:31.657 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:31.657 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:31.657 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:31.658 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:31.658 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:31.658 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:31.658 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:31.658 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:31.658 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:31.658 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:31.658 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:35.661 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:35.662 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:35.662 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:35.662 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:35.662 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:35.662 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:35.662 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:35.663 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:35.663 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:35.663 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:35.663 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:35.663 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:35.663 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:35.663 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:39.664 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:39.665 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:39.665 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:39.665 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:39.665 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:39.665 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:39.665 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:39.666 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:39.666 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:39.666 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:39.666 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:39.666 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:39.666 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:39.666 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:41.139 7f0dd14d4e00 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:43.668 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:43.669 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:43.669 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:43.669 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:43.669 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:43.669 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:43.669 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:43.670 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:43.670 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:43.670 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:43.670 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:43.670 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:43.670 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:43.670 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:47.674 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:47.675 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:47.675 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:47.675 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:47.675 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:47.675 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:47.675 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:47.676 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:47.676 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:47.676 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:47.676 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:47.676 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:47.676 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:47.676 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:51.677 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:51.678 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:51.678 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:51.678 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:51.679 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:51.679 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:51.679 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:51.679 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:51.679 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:51.679 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:51.679 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:51.679 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:51.679 7f0dd14d4e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:51.680 7f0dd14d4e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:53.216 7f0dd14d4900 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:55.680 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:55.681 7f0dd14d4900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:55.681 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:55.681 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:55.681 7f0dd14d4900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:55.681 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:55.681 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:55.682 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:55.682 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:55.682 7f0dd14d4900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:55.682 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:55.682 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:55.682 7f0dd14d4900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:55.682 7f0dd14d4900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.477 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:05:59.681 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:59.684 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:59.684 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:59.684 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:59.684 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:59.684 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:59.684 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:59.684 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:59.685 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.685 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:59.685 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.685 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.685 7f0dd14c3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:59.685 7f0dd14c3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.827 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.827 7f0dd13c8c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19699616 disk_space_available_1k = 61390868 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:05:59.827 7f0dd13c8c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.993 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:06:00.789 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:06:03.686 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:03.688 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:03.688 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:03.688 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:03.688 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:03.688 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:03.688 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:03.689 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:03.689 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:03.689 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:03.689 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:03.689 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:03.689 7f0dd14c3720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:03.689 7f0dd14c3720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:03.826 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19699652, partition_space_available_1k=61390832, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1304188, cpu_share=0.03, mem_res=1295404, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46468, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9384128, free=19807908, buffers=692296, cached=2780788, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.63, five_min_avg=0.73, fifteen_min_avg=1.01, cpu_share=0.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:06:03.826 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:03.827 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19699652 partition_space_available_1k = 61390832 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1304188 cpu_share = 0.03 mem_res = 1295404 >>contrail-database-nodemgr : << mem_virt = 46468 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 9384128 free = 19807908 buffers = 692296 cached = 2780788 node_type = database-node >> system_cpu_usage = << one_min_avg = 0.63 five_min_avg = 0.73 fifteen_min_avg = 1.01 cpu_share = 0.88 node_type = database-node >> >> 09/01/2025 02:06:03.827 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:05.386 7f0dd13c8ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:06:07.689 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:07.690 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:07.690 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:07.690 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:07.690 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:07.690 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:07.691 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:07.691 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:07.691 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:07.691 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:07.691 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:07.691 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:07.691 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:07.691 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:11.694 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:11.695 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:11.695 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:11.695 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:11.695 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:11.695 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:11.695 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:11.696 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:11.696 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:11.696 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:11.696 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:11.696 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:11.696 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:11.696 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:15.699 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:15.700 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:15.700 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:15.700 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:15.700 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:15.701 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:15.701 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:15.701 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:15.701 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:15.701 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:15.701 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:15.701 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:15.702 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:15.702 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:17.471 7f0dd13c8ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:06:19.702 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:19.704 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:19.704 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:19.704 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:19.704 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:19.704 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:19.704 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:19.705 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:19.705 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:19.705 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:19.705 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:19.705 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:19.705 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:19.705 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:23.705 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:23.706 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:23.706 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:23.706 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:23.707 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:23.707 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:23.707 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:23.707 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:23.707 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:23.707 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:23.707 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:23.707 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:23.708 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:23.708 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:27.711 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:27.712 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:27.712 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:27.713 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:27.713 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:27.713 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:27.713 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:27.713 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:27.713 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:27.713 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:27.713 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:27.713 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:27.714 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:27.714 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:29.928 7f0dd13c8ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:06:31.714 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:31.715 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:31.716 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:31.716 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:31.716 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:31.716 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:31.716 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:31.716 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:31.716 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:31.716 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:31.717 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:31.717 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:31.717 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:31.717 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:31.999 7f0dd13c8ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:06:34.099 7f0dd13c8ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:06:35.718 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:35.719 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:35.720 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:35.720 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:35.720 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:35.720 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:35.720 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:35.721 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:35.721 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:35.721 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:35.721 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:35.721 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:35.721 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:35.721 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:39.721 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:39.722 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:39.722 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:39.722 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:39.723 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:39.723 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:39.723 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:39.723 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:39.723 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:39.723 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:39.723 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:39.723 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:39.724 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:39.724 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:43.726 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:43.727 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:43.727 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:43.727 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:43.727 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:43.727 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:43.727 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:43.728 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:43.728 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:43.728 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:43.728 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:43.728 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:43.728 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:43.728 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:47.731 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:47.732 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:47.733 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:47.733 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:47.733 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:47.733 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:47.733 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:47.733 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:47.733 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:47.733 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:47.734 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:47.734 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:47.734 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:47.734 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:51.737 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:51.738 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:51.738 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:51.738 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:51.739 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:51.739 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:51.739 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:51.739 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:51.739 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:51.739 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:51.739 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:51.739 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:51.739 7f0dd13c8b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:51.740 7f0dd13c8b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:55.740 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:55.741 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:55.741 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:55.741 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:55.741 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:55.741 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:55.741 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:55.742 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:55.742 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:55.742 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:55.742 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:55.742 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:55.742 7f0dd14c37c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:55.742 7f0dd14c37c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:59.470 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:06:59.742 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:59.745 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:59.746 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:59.746 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:59.746 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:59.746 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:59.746 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:59.746 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:59.747 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:59.747 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:59.747 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:59.747 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:59.747 7f0dd14c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:59.747 7f0dd14c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:59.823 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:59.824 7f0dd1659d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. database_usage = [ << disk_space_used_1k = 19716140 disk_space_available_1k = 61374344 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:06:59.824 7f0dd1659d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:59.994 7f0dd425b280 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:07:00.320 7f0dd13c8ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:07:00.820 7f0dd425b280 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:07:03.748 7f0dd14d4860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:03.749 7f0dd14d4860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:03.750 7f0dd14d4860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:03.750 7f0dd14d4860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:03.750 7f0dd14d4860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:03.750 7f0dd14d4860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:03.750 7f0dd14d4860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:03.750 7f0dd14d4860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:03.750 7f0dd14d4860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:03.751 7f0dd14d4860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:03.751 7f0dd14d4860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:03.751 7f0dd14d4860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:03.751 7f0dd14d4860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691458801009 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:03.751 7f0dd14d4860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:03.848 7f0dd425b280 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3994-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19715948, partition_space_available_1k=61374536, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1304576, cpu_share=0.03, mem_res=1295604, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46560, cpu_share=0.02, mem_res=43016, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9405816, free=19584524, buffers=692488, cached=2982292, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.69, five_min_avg=0.74, fifteen_min_avg=0.99, cpu_share=1.12, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:07:03.849 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:03.849 7f0dd13c8ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3994-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19715948 partition_space_available_1k = 61374536 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1304576 cpu_share = 0.03 mem_res = 1295604 >>contrail-database-nodemgr : << mem_virt = 46560 cpu_share = 0.02 mem_res = 43016 >> } system_mem_usage = << total = 32665120 used = 9405816 free = 19584524 buffers = 692488 cached = 2982292 node_type = database-node >> system_cpu_usage = << one_min_avg = 0.69 five_min_avg = 0.74 fifteen_min_avg = 0.99 cpu_share = 1.12 node_type = database-node >> >> 09/01/2025 02:07:03.849 7f0dd13c8ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]