06/02/2025 02:06:05.351 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 06/02/2025 02:06:05.351 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 06/02/2025 02:06:05.351 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 06/02/2025 02:06:05.365 7f3bf725f048 [contrail-analytics-alarm-nodemgr] [ERROR]: Cannot write http_port 8113 to /tmp/contrail-analytics-alarm-nodemgr.0.http_port 06/02/2025 02:06:05.367 7f3bf725f048 [contrail-analytics-alarm-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8113 06/02/2025 02:06:05.368 7f3bf7264978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvStart] in state[none] 06/02/2025 02:06:05.369 7f3bf7264978 [contrail-analytics-alarm-nodemgr] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:06:05.371 7f3bf7264978 [contrail-analytics-alarm-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/02/2025 02:06:05.371 7f3bf7264978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 06/02/2025 02:06:05.372 7f3bf7264978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:05.373 7f3bf7264978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/02/2025 02:06:05.373 7f3bf7264978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:05.373 7f3bf7264978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:05.373 7f3bf7264978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 06/02/2025 02:06:05.373 7f3bf7264978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:05.431 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'contrail-alarm-gen', 'group': 'contrail-alarm-gen', 'pid': 45806944978843400559459810574387053249431748144702053449922237835286665297767, 'start': '1748829962000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'kafka', 'group': 'kafka', 'pid': 64177701477960671075698726090715733645584061121149135617517816336578334213920, 'start': '1748829962000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-analytics-alarm-nodemgr', 'group': 'contrail-analytics-alarm-nodemgr', 'pid': 9617950443570741936861521085318186052043773900283914315561908465267899563409, 'start': '1748829962000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 06/02/2025 02:06:05.627 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-analytics-alarm-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-542-1', _node_type='AnalyticsAlarm', _instance_id='0', _timestamp=1748829965627569, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-analytics-alarm-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo'), _seqnum=0) 06/02/2025 02:06:05.628 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-analytics-alarm-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-542-1', _node_type='AnalyticsAlarm', _instance_id='0', _timestamp=1748829965628339, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-alarm-gen', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829962000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='kafka', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829962000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-analytics-alarm-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829962000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-02 00:52:12.596288", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9311-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo'), _seqnum=0) 06/02/2025 02:06:05.631 7f3bf7108630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:05.632 7f3bf7108630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-02 00:52:12.596288", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9311-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 06/02/2025 02:06:05.632 7f3bf7108630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:05.633 7f3bf7108630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:05.633 7f3bf7108630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/02/2025 02:06:05.633 7f3bf7108630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:05.633 7f3bf7108630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:05.634 7f3bf7108630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 deleted = False process_info = [ << process_name = contrail-alarm-gen process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829962000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = kafka process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829962000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-analytics-alarm-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829962000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-02 00:52:12.596288", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9311-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 06/02/2025 02:06:05.634 7f3bf7108630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:09.370 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:06:10.664 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-analytics-alarm-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-542-1', _node_type='AnalyticsAlarm', _instance_id='0', _timestamp=1748829970664872, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-alarm-gen', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829962000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='kafka', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829962000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-analytics-alarm-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829962000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-02 00:52:12.596288", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9311-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo'), _seqnum=0) 06/02/2025 02:06:10.665 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51062292, partition_space_available_1k=30028192, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1760, cpu_share=0.0, mem_res=804, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=205504, cpu_share=0.0, mem_res=129788, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50624, cpu_share=0.0, mem_res=39364, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10206132, free=634892, buffers=959432, cached=20864664, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.41, five_min_avg=3.38, fifteen_min_avg=2.46, cpu_share=0, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:06:11.292 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:06:14.376 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:06:14.376 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:06:14.377 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:14.377 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:06:14.377 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:14.377 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:14.377 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 deleted = False process_info = [ << process_name = contrail-alarm-gen process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829962000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = kafka process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829962000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-analytics-alarm-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829962000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-02 00:52:12.596288", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9311-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 06/02/2025 02:06:14.377 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:14.377 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:14.377 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51062292 partition_space_available_1k = 30028192 percentage_partition_space_used = 63 >> } all_core_file_list = [ ] process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 1760 cpu_share = 0.0 mem_res = 804 >>kafka : << mem_virt = 205504 cpu_share = 0.0 mem_res = 129788 >>contrail-analytics-alarm-nodemgr : << mem_virt = 50624 cpu_share = 0.0 mem_res = 39364 >> } system_mem_usage = << total = 32665120 used = 10206132 free = 634892 buffers = 959432 cached = 20864664 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 4.41 five_min_avg = 3.38 fifteen_min_avg = 2.46 cpu_share = 0 node_type = analytics-alarm-node >> >> 06/02/2025 02:06:14.377 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:14.377 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:06:14.378 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:06:14.378 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:14.378 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:06:14.378 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:14.378 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:14.378 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:06:14.378 7f3bf7caff60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:18.380 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:06:20.824 7f3bf7278198 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:06:23.387 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:06:23.387 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:06:23.387 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:23.388 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:06:23.389 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:23.389 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:06:23.390 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:06:23.390 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:23.390 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:06:23.390 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:23.390 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:23.390 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:06:23.390 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:27.391 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:06:31.862 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:06:32.395 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:06:32.395 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:06:32.395 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:32.395 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:06:32.396 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:32.396 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:06:32.397 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:06:32.397 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:32.397 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:06:32.397 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:32.397 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:32.397 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:06:32.397 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:36.395 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:06:41.405 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:06:41.405 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:06:41.405 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:41.406 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:06:41.406 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:41.406 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:06:41.406 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:06:41.407 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:41.407 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:06:41.407 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:41.407 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:41.408 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:06:41.409 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:45.408 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:06:50.414 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:06:50.415 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:06:50.415 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:50.415 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:06:50.415 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:50.416 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:06:50.416 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:06:50.416 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:50.416 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:06:50.417 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:50.417 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:50.418 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:06:50.418 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:52.419 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:06:54.417 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:06:59.427 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:06:59.427 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:06:59.427 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:59.427 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:06:59.427 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:59.427 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:06:59.428 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:06:59.428 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:59.428 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:06:59.428 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:59.428 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:59.428 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:06:59.428 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:03.431 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:04.736 7f3bf7cd1d68 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:07:08.440 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:08.440 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:08.440 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:08.441 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:08.441 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:08.441 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:08.443 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:08.443 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:08.443 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:08.443 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:08.443 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:08.443 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:07:08.443 7f3bf7ca48d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:11.549 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51214620, partition_space_available_1k=29875864, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2456, cpu_share=0.0, mem_res=804, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=617536, cpu_share=0.29, mem_res=530688, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=41852, cpu_share=0.01, mem_res=39492, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=13809376, free=12743208, buffers=959608, cached=5152928, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=9.09, five_min_avg=4.77, fifteen_min_avg=2.99, cpu_share=8.0, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:07:11.549 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:11.549 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51214620 partition_space_available_1k = 29875864 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 2456 cpu_share = 0.0 mem_res = 804 >>kafka : << mem_virt = 617536 cpu_share = 0.29 mem_res = 530688 >>contrail-analytics-alarm-nodemgr : << mem_virt = 41852 cpu_share = 0.01 mem_res = 39492 >> } system_mem_usage = << total = 32665120 used = 13809376 free = 12743208 buffers = 959608 cached = 5152928 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 9.09 five_min_avg = 4.77 fifteen_min_avg = 2.99 cpu_share = 8.0 node_type = analytics-alarm-node >> >> 06/02/2025 02:07:11.549 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:12.441 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:17.446 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:17.447 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:17.447 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:17.447 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:17.447 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:17.447 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:17.447 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:17.448 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:17.448 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:17.448 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:17.448 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:17.448 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:07:17.448 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:18.140 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:07:21.449 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:26.457 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:26.457 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:26.457 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:26.457 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:26.457 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:26.457 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:26.458 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:26.458 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:26.458 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:26.458 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:26.458 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:26.459 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:07:26.459 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:30.461 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:35.478 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:35.478 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:35.478 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:35.478 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:35.478 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:35.478 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:35.479 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:35.479 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:35.479 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:35.479 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:35.479 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:35.479 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:07:35.479 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:39.478 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:40.735 7f3bf66db828 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:07:44.485 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:44.485 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:44.485 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:44.485 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:44.486 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:44.486 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:44.486 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:44.487 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:44.487 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:44.487 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:44.487 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:44.487 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:07:44.487 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:48.489 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:53.492 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:53.492 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:53.492 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:53.493 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:53.493 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:53.493 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:53.493 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:53.493 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:53.493 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:53.493 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:53.494 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:53.494 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:07:53.494 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:57.494 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:02.502 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:02.502 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:02.502 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:02.502 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:02.502 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:02.502 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:02.503 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:02.503 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:02.503 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:02.503 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:02.503 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:02.504 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:08:02.504 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:06.503 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:10.665 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51255968, partition_space_available_1k=29834516, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=117548, cpu_share=0.37, mem_res=107296, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=658312, cpu_share=0.1, mem_res=570484, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=41852, cpu_share=0.01, mem_res=39548, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=14823916, free=10581980, buffers=987932, cached=6271292, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=18.84, five_min_avg=8.97, fifteen_min_avg=4.59, cpu_share=10.0, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:08:11.506 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:11.506 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:11.506 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:11.506 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:11.506 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:11.506 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:11.507 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51255968 partition_space_available_1k = 29834516 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 117548 cpu_share = 0.37 mem_res = 107296 >>kafka : << mem_virt = 658312 cpu_share = 0.1 mem_res = 570484 >>contrail-analytics-alarm-nodemgr : << mem_virt = 41852 cpu_share = 0.01 mem_res = 39548 >> } system_mem_usage = << total = 32665120 used = 14823916 free = 10581980 buffers = 987932 cached = 6271292 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 18.84 five_min_avg = 8.97 fifteen_min_avg = 4.59 cpu_share = 10.0 node_type = analytics-alarm-node >> >> 06/02/2025 02:08:11.507 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:11.507 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:11.507 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:11.507 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:11.508 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:11.508 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:11.508 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:11.508 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:08:11.508 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:15.274 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:08:15.506 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:20.513 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:20.513 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:20.513 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:20.513 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:20.514 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:20.514 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:20.514 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:20.514 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:20.514 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:20.514 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:20.514 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:20.515 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:08:20.515 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:24.514 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:29.521 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:29.521 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:29.521 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:29.522 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:29.522 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:29.522 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:29.522 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:29.522 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:29.522 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:29.523 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:29.523 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:29.523 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:08:29.523 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:30.529 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:08:33.523 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:38.526 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:38.526 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:38.526 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:38.526 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:38.526 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:38.526 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:38.527 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:38.527 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:38.527 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:38.527 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:38.527 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:38.527 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:08:38.527 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:42.529 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:47.535 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:47.535 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:47.535 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:47.535 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:47.535 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:47.535 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:47.536 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:47.536 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:47.536 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:47.536 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:47.536 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:47.536 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:08:47.536 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:51.537 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:52.262 7f3bf66e8cc0 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:08:56.541 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:56.541 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:56.541 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:56.541 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:56.542 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:56.542 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:56.542 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:56.542 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:56.543 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:56.543 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:56.543 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:56.543 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:08:56.543 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:00.544 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:02.692 7f3bf6707240 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:09:05.549 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:05.549 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:05.549 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:05.549 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:05.549 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:05.549 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:05.549 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:05.550 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:05.550 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:05.550 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:05.550 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:05.550 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:09:05.550 7f3bf653f6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:09.549 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:10.666 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51277312, partition_space_available_1k=29813172, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=117532, cpu_share=0.64, mem_res=107712, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=662968, cpu_share=0.07, mem_res=576976, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=41872, cpu_share=0.01, mem_res=39564, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15121108, free=11332628, buffers=990816, cached=5220568, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=11.63, five_min_avg=8.74, fifteen_min_avg=4.78, cpu_share=5.0, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:09:14.557 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:14.557 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:14.557 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:14.557 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:14.557 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:14.557 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:14.558 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51277312 partition_space_available_1k = 29813172 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 117532 cpu_share = 0.64 mem_res = 107712 >>kafka : << mem_virt = 662968 cpu_share = 0.07 mem_res = 576976 >>contrail-analytics-alarm-nodemgr : << mem_virt = 41872 cpu_share = 0.01 mem_res = 39564 >> } system_mem_usage = << total = 32665120 used = 15121108 free = 11332628 buffers = 990816 cached = 5220568 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 11.63 five_min_avg = 8.74 fifteen_min_avg = 4.78 cpu_share = 5.0 node_type = analytics-alarm-node >> >> 06/02/2025 02:09:14.558 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:14.559 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:14.559 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:14.560 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:14.560 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:14.560 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:14.560 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:14.560 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:09:14.560 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:18.561 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:23.565 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:23.565 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:23.565 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:23.565 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:23.565 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:23.565 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:23.566 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:23.566 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:23.566 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:23.566 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:23.566 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:23.566 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:09:23.566 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:27.569 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:32.573 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:32.573 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:32.574 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:32.574 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:32.574 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:32.574 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:32.574 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:32.574 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:32.574 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:32.574 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:32.575 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:32.575 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:09:32.575 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:36.577 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:41.582 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:41.582 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:41.582 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:41.582 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:41.582 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:41.582 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:41.583 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:41.583 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:41.583 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:41.583 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:41.583 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:41.583 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:09:41.583 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:45.584 7f3bf66ce6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:50.590 7f3bf66ce6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:50.590 7f3bf66ce6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:50.590 7f3bf66ce6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:50.590 7f3bf66ce6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:50.591 7f3bf66ce6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:50.591 7f3bf66ce6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:50.591 7f3bf66ce6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:50.591 7f3bf66ce6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:50.591 7f3bf66ce6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:50.591 7f3bf66ce6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:50.591 7f3bf66ce6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:50.592 7f3bf66ce6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:09:50.592 7f3bf66ce6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:54.591 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:59.598 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:59.599 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:59.599 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:59.599 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:59.599 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:59.599 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:59.599 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:59.600 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:59.600 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:59.600 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:59.600 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:59.600 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:09:59.600 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:03.603 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:08.605 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:08.605 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:08.605 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:08.606 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:08.606 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:08.606 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:08.607 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:08.607 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:08.607 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:08.608 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:08.608 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:08.608 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:10:08.608 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:10.663 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51287864, partition_space_available_1k=29802620, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=121412, cpu_share=0.77, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=668512, cpu_share=0.05, mem_res=580248, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=41924, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15092404, free=11191476, buffers=990888, cached=5390352, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.01, five_min_avg=7.67, fifteen_min_avg=4.66, cpu_share=3.75, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:10:10.664 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:10.664 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51287864 partition_space_available_1k = 29802620 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 121412 cpu_share = 0.77 mem_res = 107740 >>kafka : << mem_virt = 668512 cpu_share = 0.05 mem_res = 580248 >>contrail-analytics-alarm-nodemgr : << mem_virt = 41924 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15092404 free = 11191476 buffers = 990888 cached = 5390352 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 6.01 five_min_avg = 7.67 fifteen_min_avg = 4.66 cpu_share = 3.75 node_type = analytics-alarm-node >> >> 06/02/2025 02:10:10.664 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:12.607 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:17.613 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:17.613 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:17.613 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:17.613 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:17.613 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:17.613 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:17.614 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:17.614 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:17.614 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:17.614 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:17.614 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:17.615 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:10:17.615 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:21.617 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:23.697 7f3bf7108e10 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:10:26.621 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:26.621 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:26.621 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:26.621 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:26.621 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:26.621 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:26.622 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:26.622 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:26.622 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:26.622 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:26.622 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:26.622 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:10:26.622 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:30.625 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:34.357 7f3bf7cc4588 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:10:35.628 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:35.628 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:35.629 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:35.629 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:35.629 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:35.629 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:35.630 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:35.630 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:35.630 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:35.630 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:35.630 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:35.630 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:10:35.630 7f3bfad8eb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:39.629 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:44.635 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:44.635 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:44.635 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:44.635 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:44.635 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:44.636 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:44.637 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:44.637 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:44.637 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:44.637 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:44.637 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:44.637 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:10:44.637 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:48.637 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:53.641 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:53.641 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:53.641 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:53.641 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:53.641 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:53.641 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:53.642 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:53.642 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:53.642 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:53.642 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:53.642 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:53.642 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:10:53.642 7f3bfae050f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:57.644 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:02.649 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:02.650 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:02.650 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:02.650 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:02.650 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:02.650 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:02.650 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:02.650 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:02.650 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:02.650 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:02.651 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:02.651 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:11:02.651 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:05.630 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:11:06.651 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:10.682 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51301076, partition_space_available_1k=29789408, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=125384, cpu_share=0.75, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=671192, cpu_share=0.04, mem_res=583060, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=41984, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15113484, free=11132060, buffers=991572, cached=5428004, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.19, five_min_avg=6.8, fifteen_min_avg=4.55, cpu_share=3.12, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:11:11.655 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:11.655 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:11.655 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:11.655 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:11.655 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:11.655 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:11.656 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51301076 partition_space_available_1k = 29789408 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 125384 cpu_share = 0.75 mem_res = 107740 >>kafka : << mem_virt = 671192 cpu_share = 0.04 mem_res = 583060 >>contrail-analytics-alarm-nodemgr : << mem_virt = 41984 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15113484 free = 11132060 buffers = 991572 cached = 5428004 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 4.19 five_min_avg = 6.8 fifteen_min_avg = 4.55 cpu_share = 3.12 node_type = analytics-alarm-node >> >> 06/02/2025 02:11:11.656 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:11.656 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:11.656 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:11.656 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:11.657 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:11.657 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:11.657 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:11.657 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:11:11.657 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:15.657 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:20.661 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:20.662 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:20.662 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:20.662 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:20.662 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:20.662 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:20.663 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:20.663 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:20.663 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:20.663 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:20.663 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:20.664 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:11:20.664 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:24.666 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:29.672 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:29.672 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:29.672 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:29.672 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:29.672 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:29.672 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:29.673 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:29.673 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:29.673 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:29.673 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:29.673 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:29.673 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:11:29.673 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:33.673 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:38.677 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:38.678 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:38.678 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:38.678 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:38.678 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:38.678 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:38.678 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:38.679 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:38.679 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:38.679 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:38.679 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:38.679 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:11:38.679 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:42.681 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:47.686 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:47.687 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:47.687 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:47.687 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:47.687 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:47.688 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:47.688 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:47.689 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:47.689 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:47.689 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:47.689 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:47.689 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:11:47.689 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:51.690 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:55.982 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:11:56.693 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:56.693 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:56.693 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:56.693 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:56.693 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:56.693 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:56.694 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:56.694 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:56.694 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:56.694 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:56.694 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:56.694 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:11:56.694 7f3bf7cab198 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:00.696 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:05.699 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:05.699 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:05.700 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:05.700 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:05.700 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:05.700 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:05.701 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:05.701 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:05.701 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:05.701 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:05.701 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:05.701 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:12:05.701 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:09.701 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:10.681 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51303324, partition_space_available_1k=29787160, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=129364, cpu_share=0.74, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=674488, cpu_share=0.08, mem_res=586180, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42004, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15513580, free=10355420, buffers=991980, cached=5804140, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.16, five_min_avg=6.3, fifteen_min_avg=4.52, cpu_share=3.25, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:12:14.707 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:14.708 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:14.708 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:14.708 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:14.719 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:14.719 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:14.720 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51303324 partition_space_available_1k = 29787160 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 129364 cpu_share = 0.74 mem_res = 107740 >>kafka : << mem_virt = 674488 cpu_share = 0.08 mem_res = 586180 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42004 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15513580 free = 10355420 buffers = 991980 cached = 5804140 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 4.16 five_min_avg = 6.3 fifteen_min_avg = 4.52 cpu_share = 3.25 node_type = analytics-alarm-node >> >> 06/02/2025 02:12:14.726 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:14.727 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:14.728 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:14.728 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:14.728 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:14.728 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:14.729 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:14.729 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:12:14.729 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:18.712 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:23.717 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:23.717 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:23.717 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:23.717 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:23.717 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:23.718 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:23.718 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:23.718 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:23.718 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:23.718 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:23.718 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:23.718 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:12:23.719 7f3bf66e6390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:27.719 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:32.725 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:32.725 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:32.725 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:32.725 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:32.725 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:32.725 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:32.725 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:32.726 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:32.726 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:32.726 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:32.726 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:32.726 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:12:32.726 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:36.729 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:41.732 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:41.732 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:41.732 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:41.732 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:41.732 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:41.732 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:41.733 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:41.733 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:41.733 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:41.733 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:41.733 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:41.733 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:12:41.733 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:45.733 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:50.737 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:50.737 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:50.737 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:50.737 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:50.737 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:50.737 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:50.738 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:50.738 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:50.738 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:50.738 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:50.738 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:50.738 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:12:50.738 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:54.741 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:59.747 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:59.747 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:59.747 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:59.747 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:59.747 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:59.747 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:59.748 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:59.748 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:59.748 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:59.748 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:59.748 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:59.748 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:12:59.748 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:03.752 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:08.753 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:08.754 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:08.754 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:08.754 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:08.754 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:08.754 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:08.755 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:08.755 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:08.755 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:08.755 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:08.755 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:08.755 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:13:08.755 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:10.672 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51315488, partition_space_available_1k=29774996, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=133416, cpu_share=0.65, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=676272, cpu_share=0.03, mem_res=587976, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42004, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15138664, free=9733768, buffers=991988, cached=6800700, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.29, five_min_avg=6.53, fifteen_min_avg=4.72, cpu_share=4.88, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:13:10.672 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:10.672 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51315488 partition_space_available_1k = 29774996 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 133416 cpu_share = 0.65 mem_res = 107740 >>kafka : << mem_virt = 676272 cpu_share = 0.03 mem_res = 587976 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42004 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15138664 free = 9733768 buffers = 991988 cached = 6800700 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 5.29 five_min_avg = 6.53 fifteen_min_avg = 4.72 cpu_share = 4.88 node_type = analytics-alarm-node >> >> 06/02/2025 02:13:10.674 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:12.755 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:17.761 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:17.762 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:17.762 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:17.762 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:17.762 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:17.762 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:17.763 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:17.763 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:17.763 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:17.763 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:17.764 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:17.764 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:13:17.764 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:21.764 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:26.768 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:26.768 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:26.768 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:26.768 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:26.768 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:26.768 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:26.769 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:26.769 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:26.769 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:26.769 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:26.769 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:26.769 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:13:26.769 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:30.772 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:35.778 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:35.778 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:35.778 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:35.779 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:35.779 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:35.779 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:35.779 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:35.779 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:35.779 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:35.780 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:35.780 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:35.780 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:13:35.780 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:39.783 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:44.788 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:44.789 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:44.789 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:44.789 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:44.789 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:44.789 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:44.790 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:44.790 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:44.790 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:44.790 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:44.790 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:44.790 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:13:44.790 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:48.792 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:53.797 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:53.797 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:53.797 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:53.797 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:53.797 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:53.797 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:53.798 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:53.798 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:53.798 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:53.798 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:53.798 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:53.799 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:13:53.799 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:57.798 7f3bf66ff978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:02.804 7f3bf66ff978 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:02.805 7f3bf66ff978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:02.805 7f3bf66ff978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:02.805 7f3bf66ff978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:02.805 7f3bf66ff978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:02.805 7f3bf66ff978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:02.806 7f3bf66ff978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:02.806 7f3bf66ff978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:02.806 7f3bf66ff978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:02.806 7f3bf66ff978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:02.806 7f3bf66ff978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:02.806 7f3bf66ff978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:14:02.806 7f3bf66ff978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:06.805 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:10.681 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51322072, partition_space_available_1k=29768412, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=137364, cpu_share=0.8, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=676212, cpu_share=0.03, mem_res=588020, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42060, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15164612, free=9701564, buffers=992000, cached=6806944, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.69, five_min_avg=5.86, fifteen_min_avg=4.61, cpu_share=2.62, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:14:11.808 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:11.809 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:11.809 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:11.809 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:11.809 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:11.809 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:11.809 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51322072 partition_space_available_1k = 29768412 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 137364 cpu_share = 0.8 mem_res = 107740 >>kafka : << mem_virt = 676212 cpu_share = 0.03 mem_res = 588020 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42060 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15164612 free = 9701564 buffers = 992000 cached = 6806944 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 3.69 five_min_avg = 5.86 fifteen_min_avg = 4.61 cpu_share = 2.62 node_type = analytics-alarm-node >> >> 06/02/2025 02:14:11.809 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:11.810 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:11.810 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:11.810 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:11.811 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:11.811 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:11.811 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:11.811 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:14:11.811 7f3bf7ca4978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:15.812 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:20.819 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:20.819 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:20.819 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:20.819 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:20.819 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:20.819 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:20.820 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:20.820 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:20.820 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:20.820 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:20.820 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:20.820 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:14:20.820 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:24.823 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:29.829 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:29.829 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:29.829 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:29.829 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:29.829 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:29.830 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:29.830 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:29.830 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:29.830 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:29.830 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:29.831 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:29.831 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:14:29.831 7f3bf66ff240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:33.832 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:38.836 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:38.837 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:38.837 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:38.837 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:38.837 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:38.837 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:38.838 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:38.838 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:38.838 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:38.838 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:38.838 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:38.838 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:14:38.838 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:42.838 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:47.845 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:47.845 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:47.845 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:47.845 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:47.845 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:47.845 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:47.846 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:47.846 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:47.846 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:47.846 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:47.846 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:47.846 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:14:47.846 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:51.849 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:56.850 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:56.850 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:56.851 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:56.851 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:56.851 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:56.851 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:56.851 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:56.851 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:56.851 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:56.851 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:56.852 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:56.852 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:14:56.852 7f3bfae05b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:00.854 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:05.856 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:05.856 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:05.857 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:05.857 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:05.857 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:05.857 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:05.858 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:05.858 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:05.858 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:05.858 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:05.858 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:05.858 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:15:05.858 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:09.858 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:10.025 7f3bf89b1048 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:15:10.686 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51333744, partition_space_available_1k=29756740, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=141392, cpu_share=0.9, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=684512, cpu_share=0.04, mem_res=595672, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42080, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15206336, free=9645224, buffers=992420, cached=6821140, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.66, five_min_avg=5.15, fifteen_min_avg=4.44, cpu_share=3.0, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:15:14.862 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:14.862 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:14.863 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:14.863 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:14.863 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:14.863 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:14.863 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51333744 partition_space_available_1k = 29756740 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 141392 cpu_share = 0.9 mem_res = 107740 >>kafka : << mem_virt = 684512 cpu_share = 0.04 mem_res = 595672 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42080 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15206336 free = 9645224 buffers = 992420 cached = 6821140 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.66 five_min_avg = 5.15 fifteen_min_avg = 4.44 cpu_share = 3.0 node_type = analytics-alarm-node >> >> 06/02/2025 02:15:14.863 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:14.863 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:14.864 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:14.864 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:14.864 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:14.864 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:14.864 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:14.864 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:15:14.864 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:18.865 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:20.338 7f3bf7cc4828 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:15:23.870 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:23.870 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:23.870 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:23.870 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:23.870 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:23.870 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:23.871 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:23.871 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:23.871 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:23.871 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:23.871 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:23.871 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:15:23.871 7f3bf7275978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:27.871 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:31.089 7f3bf66fd828 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:15:32.874 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:32.875 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:32.875 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:32.875 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:32.875 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:32.875 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:32.876 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:32.876 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:32.876 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:32.876 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:32.876 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:32.876 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:15:32.876 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:36.878 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:41.885 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:41.885 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:41.885 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:41.885 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:41.885 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:41.885 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:41.886 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:41.886 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:41.886 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:41.886 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:41.886 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:41.886 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:15:41.886 7f3bf66ff8d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:45.889 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:50.895 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:50.895 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:50.896 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:50.896 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:50.896 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:50.896 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:50.897 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:50.897 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:50.897 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:50.897 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:50.897 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:50.897 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:15:50.897 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:54.898 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:59.902 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:59.902 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:59.902 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:59.902 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:59.902 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:59.902 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:59.903 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:59.903 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:59.903 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:59.903 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:59.903 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:59.903 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:15:59.903 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:03.906 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:08.908 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:08.908 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:08.908 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:08.909 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:08.909 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:08.909 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:08.909 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:08.909 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:08.910 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:08.910 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:08.910 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:08.910 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:16:08.910 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:10.694 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51337412, partition_space_available_1k=29753072, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=145608, cpu_share=0.83, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=683860, cpu_share=0.04, mem_res=594984, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42072, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15159260, free=9672688, buffers=993384, cached=6839788, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.38, five_min_avg=4.66, fifteen_min_avg=4.32, cpu_share=3.12, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:16:10.694 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:10.694 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51337412 partition_space_available_1k = 29753072 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 145608 cpu_share = 0.83 mem_res = 107740 >>kafka : << mem_virt = 683860 cpu_share = 0.04 mem_res = 594984 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42072 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15159260 free = 9672688 buffers = 993384 cached = 6839788 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.38 five_min_avg = 4.66 fifteen_min_avg = 4.32 cpu_share = 3.12 node_type = analytics-alarm-node >> >> 06/02/2025 02:16:10.694 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:12.908 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:17.913 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:17.913 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:17.913 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:17.913 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:17.913 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:17.913 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:17.914 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:17.914 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:17.914 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:17.914 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:17.914 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:17.914 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:16:17.914 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:21.917 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:26.924 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:26.925 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:26.925 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:26.925 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:26.925 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:26.925 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:26.925 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:26.926 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:26.926 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:26.926 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:26.926 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:26.926 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:16:26.926 7f3bfbeb2c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:30.927 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:35.934 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:35.935 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:35.935 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:35.935 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:35.935 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:35.935 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:35.935 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:35.936 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:35.936 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:35.936 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:35.936 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:35.936 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:16:35.936 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:39.939 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:44.942 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:44.942 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:44.942 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:44.942 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:44.942 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:44.942 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:44.943 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:44.943 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:44.943 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:44.943 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:44.943 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:44.943 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:16:44.943 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:48.946 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:53.953 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:53.953 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:53.953 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:53.954 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:53.954 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:53.954 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:53.954 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:53.954 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:53.954 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:53.955 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:53.955 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:53.955 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:16:53.955 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:54.734 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:16:57.956 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:02.962 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:02.963 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:02.963 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:02.963 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:02.963 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:02.963 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:02.964 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:02.964 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:02.964 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:02.964 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:02.964 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:02.964 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:17:02.965 7f3bf7cad240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:06.963 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:10.692 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51349780, partition_space_available_1k=29740704, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=149244, cpu_share=0.57, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=681580, cpu_share=0.04, mem_res=592728, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42132, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15185084, free=9604848, buffers=994116, cached=6881072, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.04, five_min_avg=5.17, fifteen_min_avg=4.52, cpu_share=5.12, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:17:11.966 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:11.966 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:11.967 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:11.967 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:11.967 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:11.967 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:11.967 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51349780 partition_space_available_1k = 29740704 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 149244 cpu_share = 0.57 mem_res = 107740 >>kafka : << mem_virt = 681580 cpu_share = 0.04 mem_res = 592728 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42132 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15185084 free = 9604848 buffers = 994116 cached = 6881072 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 5.04 five_min_avg = 5.17 fifteen_min_avg = 4.52 cpu_share = 5.12 node_type = analytics-alarm-node >> >> 06/02/2025 02:17:11.967 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:11.967 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:11.968 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:11.968 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:11.969 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:11.969 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:11.969 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:11.969 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:17:11.969 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:15.969 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:20.975 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:20.975 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:20.975 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:20.976 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:20.976 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:20.976 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:20.976 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:20.976 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:20.977 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:20.977 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:20.977 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:20.977 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:17:20.977 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:24.978 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:29.981 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:29.981 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:29.981 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:29.981 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:29.982 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:29.982 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:29.982 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:29.982 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:29.982 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:29.982 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:29.982 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:29.982 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:17:29.983 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:33.985 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:38.989 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:38.989 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:38.989 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:38.989 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:38.990 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:38.990 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:38.990 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:38.990 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:38.991 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:38.991 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:38.991 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:38.991 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:17:38.991 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:42.993 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:47.998 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:47.998 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:47.998 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:47.999 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:47.999 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:47.999 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:47.999 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:47.999 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:47.999 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:47.999 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:47.999 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:48.000 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:17:48.000 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:52.002 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:57.008 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:57.008 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:57.008 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:57.009 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:57.009 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:57.009 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:57.009 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:57.009 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:57.010 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:57.010 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:57.010 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:57.010 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:17:57.010 7f3bf84afeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:01.010 7f3bf725f630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:06.013 7f3bf725f630 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:06.014 7f3bf725f630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:06.014 7f3bf725f630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:06.015 7f3bf725f630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:06.015 7f3bf725f630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:06.015 7f3bf725f630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:06.016 7f3bf725f630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:06.016 7f3bf725f630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:06.016 7f3bf725f630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:06.016 7f3bf725f630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:06.016 7f3bf725f630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:06.016 7f3bf725f630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:18:06.016 7f3bf725f630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:10.015 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:10.696 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51357152, partition_space_available_1k=29733332, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=153436, cpu_share=0.71, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=703468, cpu_share=0.03, mem_res=614508, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42152, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15212584, free=9569808, buffers=994116, cached=6888612, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.11, five_min_avg=4.59, fifteen_min_avg=4.36, cpu_share=2.62, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:18:15.017 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:15.018 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:15.018 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:15.018 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:15.018 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:15.018 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:15.018 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51357152 partition_space_available_1k = 29733332 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 153436 cpu_share = 0.71 mem_res = 107740 >>kafka : << mem_virt = 703468 cpu_share = 0.03 mem_res = 614508 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42152 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15212584 free = 9569808 buffers = 994116 cached = 6888612 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 3.11 five_min_avg = 4.59 fifteen_min_avg = 4.36 cpu_share = 2.62 node_type = analytics-alarm-node >> >> 06/02/2025 02:18:15.018 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:15.018 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:15.019 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:15.019 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:15.019 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:15.019 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:15.019 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:15.019 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:18:15.019 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:19.021 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:24.026 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:24.027 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:24.027 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:24.027 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:24.027 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:24.027 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:24.027 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:24.027 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:24.028 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:24.028 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:24.028 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:24.028 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:18:24.028 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:28.031 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:33.037 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:33.038 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:33.038 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:33.038 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:33.038 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:33.039 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:33.039 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:33.040 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:33.040 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:33.040 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:33.040 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:33.040 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:18:33.040 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:37.041 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:42.046 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:42.047 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:42.047 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:42.047 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:42.047 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:42.047 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:42.047 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:42.048 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:42.048 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:42.048 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:42.048 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:42.048 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:18:42.048 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:46.048 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:51.054 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:51.054 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:51.054 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:51.054 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:51.054 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:51.055 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:51.055 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:51.055 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:51.055 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:51.055 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:51.055 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:51.055 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:18:51.056 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:55.058 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:00.060 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:00.060 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:00.060 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:00.060 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:00.060 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:00.060 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:00.061 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:00.061 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:00.061 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:00.061 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:00.061 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:00.061 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:19:00.061 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:04.064 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:09.067 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:09.067 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:09.068 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:09.068 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:09.068 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:09.068 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:09.069 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:09.069 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:09.069 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:09.069 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:09.069 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:09.069 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:19:09.069 7f3bf72786d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:10.696 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51362616, partition_space_available_1k=29727868, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=157412, cpu_share=0.86, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=703468, cpu_share=0.02, mem_res=614508, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42144, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15193016, free=9582896, buffers=994124, cached=6895084, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.52, five_min_avg=4.15, fifteen_min_avg=4.22, cpu_share=2.75, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:19:10.697 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:10.697 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51362616 partition_space_available_1k = 29727868 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 157412 cpu_share = 0.86 mem_res = 107740 >>kafka : << mem_virt = 703468 cpu_share = 0.02 mem_res = 614508 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42144 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15193016 free = 9582896 buffers = 994124 cached = 6895084 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.52 five_min_avg = 4.15 fifteen_min_avg = 4.22 cpu_share = 2.75 node_type = analytics-alarm-node >> >> 06/02/2025 02:19:10.697 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:13.068 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:18.073 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:18.073 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:18.073 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:18.073 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:18.073 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:18.073 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:18.074 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:18.074 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:18.074 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:18.074 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:18.074 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:18.074 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:19:18.074 7f3bf7cad630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:22.077 7f3bf66e88d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:27.082 7f3bf66e88d0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:27.083 7f3bf66e88d0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:27.083 7f3bf66e88d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:27.083 7f3bf66e88d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:27.083 7f3bf66e88d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:27.083 7f3bf66e88d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:27.083 7f3bf66e88d0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:27.083 7f3bf66e88d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:27.084 7f3bf66e88d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:27.084 7f3bf66e88d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:27.084 7f3bf66e88d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:27.084 7f3bf66e88d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:19:27.084 7f3bf66e88d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:31.083 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:36.089 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:36.089 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:36.089 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:36.089 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:36.089 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:36.089 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:36.090 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:36.090 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:36.090 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:36.090 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:36.090 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:36.090 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:19:36.090 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:40.092 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:45.097 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:45.097 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:45.097 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:45.097 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:45.097 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:45.097 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:45.098 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:45.098 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:45.098 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:45.098 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:45.098 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:45.099 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:19:45.099 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:49.100 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:54.106 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:54.106 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:54.106 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:54.107 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:54.107 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:54.107 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:54.107 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:54.107 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:54.107 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:54.107 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:54.107 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:54.107 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:19:54.108 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:58.109 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:03.113 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:03.113 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:03.113 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:03.113 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:03.113 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:03.113 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:03.114 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:03.114 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:03.114 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:03.114 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:03.114 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:03.114 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:20:03.114 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:07.114 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:10.695 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51368076, partition_space_available_1k=29722408, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=161204, cpu_share=0.85, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=709764, cpu_share=0.02, mem_res=620584, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42200, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15201860, free=9567564, buffers=994124, cached=6901572, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.16, five_min_avg=3.77, fifteen_min_avg=4.09, cpu_share=2.75, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:20:12.117 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:12.117 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:12.117 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:12.117 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:12.117 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:12.117 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:12.117 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51368076 partition_space_available_1k = 29722408 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 161204 cpu_share = 0.85 mem_res = 107740 >>kafka : << mem_virt = 709764 cpu_share = 0.02 mem_res = 620584 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42200 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15201860 free = 9567564 buffers = 994124 cached = 6901572 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.16 five_min_avg = 3.77 fifteen_min_avg = 4.09 cpu_share = 2.75 node_type = analytics-alarm-node >> >> 06/02/2025 02:20:12.117 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:12.117 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:12.118 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:12.118 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:12.118 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:12.118 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:12.118 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:12.118 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:20:12.118 7f3bf7cbeb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:16.118 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:21.125 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:21.125 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:21.125 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:21.125 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:21.125 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:21.125 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:21.126 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:21.126 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:21.126 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:21.126 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:21.126 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:21.126 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:20:21.126 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:22.437 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:20:25.127 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:30.133 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:30.133 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:30.133 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:30.134 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:30.134 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:30.134 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:30.134 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:30.134 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:30.134 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:30.135 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:30.136 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:30.136 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:20:30.136 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:33.150 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:20:34.134 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:39.139 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:39.139 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:39.139 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:39.139 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:39.140 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:39.140 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:39.140 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:39.140 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:39.140 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:39.141 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:39.141 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:39.141 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:20:39.141 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:43.141 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:43.570 7f3bf89b1780 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:20:48.146 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:48.146 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:48.146 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:48.146 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:48.146 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:48.146 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:48.147 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:48.147 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:48.147 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:48.147 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:48.147 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:48.147 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:20:48.147 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:52.150 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:57.156 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:57.156 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:57.156 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:57.157 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:57.157 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:57.157 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:57.157 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:57.157 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:57.157 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:57.157 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:57.158 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:57.158 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:20:57.158 7f3bfff98f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:01.157 7f3bfae05588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:06.160 7f3bfae05588 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:06.161 7f3bfae05588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:06.161 7f3bfae05588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:06.161 7f3bfae05588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:06.161 7f3bfae05588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:06.161 7f3bfae05588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:06.162 7f3bfae05588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:06.162 7f3bfae05588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:06.162 7f3bfae05588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:06.162 7f3bfae05588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:06.162 7f3bfae05588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:06.162 7f3bfae05588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:21:06.162 7f3bfae05588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:10.162 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:10.712 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51377604, partition_space_available_1k=29712880, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=165348, cpu_share=0.89, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=746208, cpu_share=0.03, mem_res=657200, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42224, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15234544, free=9520772, buffers=995508, cached=6914296, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.23, five_min_avg=3.5, fifteen_min_avg=3.97, cpu_share=3.12, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:21:15.168 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:15.168 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:15.168 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:15.169 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:15.169 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:15.170 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:15.170 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51377604 partition_space_available_1k = 29712880 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 165348 cpu_share = 0.89 mem_res = 107740 >>kafka : << mem_virt = 746208 cpu_share = 0.03 mem_res = 657200 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42224 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15234544 free = 9520772 buffers = 995508 cached = 6914296 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.23 five_min_avg = 3.5 fifteen_min_avg = 3.97 cpu_share = 3.12 node_type = analytics-alarm-node >> >> 06/02/2025 02:21:15.170 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:15.170 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:15.170 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:15.170 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:15.170 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:15.171 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:15.171 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:15.171 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:21:15.171 7f3bf66ffb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:19.171 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:24.180 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:24.180 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:24.180 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:24.180 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:24.180 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:24.180 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:24.181 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:24.181 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:24.181 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:24.181 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:24.181 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:24.181 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:21:24.181 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:28.186 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:33.189 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:33.189 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:33.189 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:33.189 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:33.189 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:33.189 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:33.190 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:33.190 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:33.190 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:33.190 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:33.190 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:33.190 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:21:33.190 7f3bf72788d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:37.193 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:42.200 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:42.200 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:42.200 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:42.200 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:42.200 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:42.200 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:42.201 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:42.201 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:42.201 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:42.201 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:42.201 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:42.201 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:21:42.201 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:46.200 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:51.207 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:51.207 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:51.207 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:51.207 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:51.207 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:51.207 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:51.208 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:51.208 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:51.208 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:51.208 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:51.208 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:51.208 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:21:51.208 7f3bf7caf240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:52.139 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:21:55.209 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:00.216 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:00.216 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:00.216 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:00.216 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:00.216 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:00.217 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:00.217 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:00.217 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:00.217 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:00.217 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:00.217 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:00.217 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:22:00.217 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:04.217 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:09.219 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:09.220 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:09.220 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:09.220 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:09.220 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:09.220 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:09.221 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:09.221 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:09.222 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:09.222 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:09.222 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:09.222 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:22:09.222 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:10.705 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51385292, partition_space_available_1k=29705192, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=164140, cpu_share=0.66, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=746248, cpu_share=0.02, mem_res=657204, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42220, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15267132, free=9468628, buffers=996092, cached=6933268, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.96, five_min_avg=4.28, fifteen_min_avg=4.22, cpu_share=4.88, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:22:10.705 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:10.705 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51385292 partition_space_available_1k = 29705192 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 164140 cpu_share = 0.66 mem_res = 107740 >>kafka : << mem_virt = 746248 cpu_share = 0.02 mem_res = 657204 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42220 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15267132 free = 9468628 buffers = 996092 cached = 6933268 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 4.96 five_min_avg = 4.28 fifteen_min_avg = 4.22 cpu_share = 4.88 node_type = analytics-alarm-node >> >> 06/02/2025 02:22:10.705 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:13.221 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:18.224 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:18.224 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:18.224 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:18.225 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:18.225 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:18.225 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:18.225 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:18.226 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:18.226 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:18.226 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:18.226 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:18.226 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:22:18.226 7f3bf7cafa20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:22.228 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:27.234 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:27.234 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:27.234 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:27.235 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:27.235 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:27.235 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:27.235 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:27.235 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:27.235 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:27.235 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:27.235 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:27.235 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:22:27.235 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:31.238 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:36.243 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:36.244 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:36.244 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:36.244 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:36.244 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:36.244 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:36.245 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:36.245 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:36.245 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:36.245 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:36.245 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:36.245 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:22:36.245 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:40.247 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:45.253 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:45.254 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:45.254 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:45.254 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:45.254 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:45.254 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:45.255 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:45.255 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:45.255 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:45.255 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:45.255 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:45.255 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:22:45.255 7f3bf7275c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:49.256 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:54.262 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:54.262 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:54.262 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:54.262 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:54.262 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:54.262 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:54.263 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:54.263 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:54.263 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:54.263 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:54.263 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:54.264 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:22:54.264 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:58.263 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:03.265 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:03.265 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:03.266 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:03.266 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:03.266 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:03.266 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:03.266 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:03.266 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:03.266 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:03.266 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:03.266 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:03.267 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:23:03.267 7f3bf66c9d68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:07.266 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:10.706 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51392696, partition_space_available_1k=29697788, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168140, cpu_share=0.65, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=749288, cpu_share=0.02, mem_res=660140, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42272, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15277624, free=9450776, buffers=996100, cached=6940620, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.17, five_min_avg=3.93, fifteen_min_avg=4.11, cpu_share=2.38, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:23:12.270 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:12.270 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:12.270 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:12.270 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:12.270 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:12.270 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:12.270 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51392696 partition_space_available_1k = 29697788 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 168140 cpu_share = 0.65 mem_res = 107740 >>kafka : << mem_virt = 749288 cpu_share = 0.02 mem_res = 660140 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42272 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15277624 free = 9450776 buffers = 996100 cached = 6940620 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 3.17 five_min_avg = 3.93 fifteen_min_avg = 4.11 cpu_share = 2.38 node_type = analytics-alarm-node >> >> 06/02/2025 02:23:12.270 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:12.270 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:12.271 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:12.271 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:12.271 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:12.271 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:12.271 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:12.271 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:23:12.271 7f3bf7108a20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:16.273 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:21.279 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:21.279 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:21.279 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:21.279 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:21.279 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:21.279 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:21.279 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:21.280 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:21.280 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:21.280 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:21.280 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:21.280 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:23:21.280 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:25.281 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:30.287 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:30.287 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:30.287 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:30.288 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:30.288 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:30.288 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:30.289 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:30.289 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:30.289 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:30.289 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:30.289 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:30.289 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:23:30.289 7f3bfc2cdd68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:34.289 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:39.291 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:39.291 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:39.291 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:39.291 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:39.292 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:39.292 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:39.292 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:39.292 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:39.293 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:39.293 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:39.293 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:39.293 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:23:39.293 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:43.295 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:48.298 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:48.298 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:48.298 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:48.298 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:48.298 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:48.298 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:48.299 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:48.299 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:48.299 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:48.299 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:48.299 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:48.299 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:23:48.299 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:52.300 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:57.306 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:57.307 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:57.307 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:57.307 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:57.307 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:57.307 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:57.308 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:57.308 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:57.308 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:57.308 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:57.308 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:57.308 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:23:57.308 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:01.309 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:06.312 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:06.313 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:06.313 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:06.313 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:06.313 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:06.313 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:06.314 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:06.314 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:06.314 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:06.314 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:06.314 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:06.314 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:24:06.314 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:10.314 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:10.708 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51393108, partition_space_available_1k=29697376, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167156, cpu_share=0.6, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=755088, cpu_share=0.03, mem_res=666040, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42296, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15264764, free=9462508, buffers=996100, cached=6941748, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.74, five_min_avg=3.68, fifteen_min_avg=4.01, cpu_share=2.25, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:24:15.321 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:15.321 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:15.321 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:15.321 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:15.321 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:15.322 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:15.322 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51393108 partition_space_available_1k = 29697376 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 167156 cpu_share = 0.6 mem_res = 107740 >>kafka : << mem_virt = 755088 cpu_share = 0.03 mem_res = 666040 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42296 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15264764 free = 9462508 buffers = 996100 cached = 6941748 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.74 five_min_avg = 3.68 fifteen_min_avg = 4.01 cpu_share = 2.25 node_type = analytics-alarm-node >> >> 06/02/2025 02:24:15.322 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:15.322 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:15.322 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:15.322 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:15.322 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:15.322 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:15.322 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:15.323 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:24:15.323 7f3bf7cbe588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:19.322 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:24.327 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:24.328 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:24.328 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:24.328 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:24.328 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:24.328 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:24.328 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:24.328 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:24.328 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:24.329 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:24.329 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:24.329 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:24:24.329 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:28.332 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:33.334 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:33.334 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:33.334 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:33.334 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:33.334 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:33.334 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:33.335 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:33.335 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:33.335 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:33.335 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:33.335 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:33.335 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:24:33.335 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:37.338 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:42.344 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:42.344 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:42.344 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:42.344 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:42.344 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:42.345 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:42.345 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:42.345 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:42.345 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:42.345 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:42.345 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:42.345 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:24:42.345 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:46.347 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:51.353 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:51.353 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:51.353 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:51.353 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:51.354 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:51.354 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:51.354 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:51.354 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:51.354 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:51.354 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:51.355 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:51.355 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:24:51.355 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:55.357 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:00.363 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:00.363 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:00.363 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:00.364 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:00.364 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:00.364 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:00.364 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:00.364 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:00.365 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:00.365 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:00.365 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:00.365 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:25:00.365 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:04.365 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:09.366 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:09.367 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:09.367 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:09.367 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:09.367 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:09.367 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:09.368 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:09.368 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:09.368 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:09.368 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:09.368 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:09.368 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:25:09.368 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:10.704 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51393704, partition_space_available_1k=29696780, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166176, cpu_share=0.62, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=755264, cpu_share=0.02, mem_res=666044, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42288, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15263056, free=9462080, buffers=996100, cached=6943884, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.05, five_min_avg=3.29, fifteen_min_avg=3.86, cpu_share=2.5, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:25:10.704 7f3bf89b18d0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:10.704 7f3bf89b18d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51393704 partition_space_available_1k = 29696780 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 166176 cpu_share = 0.62 mem_res = 107740 >>kafka : << mem_virt = 755264 cpu_share = 0.02 mem_res = 666044 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42288 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15263056 free = 9462080 buffers = 996100 cached = 6943884 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.05 five_min_avg = 3.29 fifteen_min_avg = 3.86 cpu_share = 2.5 node_type = analytics-alarm-node >> >> 06/02/2025 02:25:10.704 7f3bf89b18d0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:13.369 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:18.371 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:18.371 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:18.371 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:18.371 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:18.372 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:18.372 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:18.373 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:18.373 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:18.373 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:18.373 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:18.373 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:18.373 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:25:18.373 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:22.375 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:27.381 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:27.382 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:27.382 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:27.382 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:27.382 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:27.382 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:27.383 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:27.383 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:27.383 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:27.383 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:27.383 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:27.383 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:25:27.383 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:31.385 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:36.391 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:36.391 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:36.391 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:36.392 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:36.392 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:36.392 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:36.392 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:36.393 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:36.393 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:36.393 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:36.393 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:36.393 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:25:36.393 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:40.396 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:45.401 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:45.401 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:45.402 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:45.402 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:45.402 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:45.402 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:45.402 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:45.402 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:45.402 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:45.402 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:45.402 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:45.403 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:25:45.403 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:49.403 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:54.409 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:54.409 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:54.409 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:54.409 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:54.409 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:54.410 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:54.410 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:54.410 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:54.410 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:54.410 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:54.410 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:54.410 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:25:54.411 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:58.413 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:03.420 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:03.420 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:03.420 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:03.420 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:03.420 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:03.421 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:03.421 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:03.421 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:03.421 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:03.421 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:03.422 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:03.422 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:26:03.422 7f3bf89b10f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:07.421 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:10.722 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51507520, partition_space_available_1k=29582964, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=169988, cpu_share=0.57, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=761680, cpu_share=0.02, mem_res=672628, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42344, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15398352, free=9391104, buffers=996124, cached=6879540, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.19, five_min_avg=3.96, fifteen_min_avg=4.05, cpu_share=5.0, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:26:12.425 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:12.425 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:12.425 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:12.426 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:12.426 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:12.426 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:12.426 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51507520 partition_space_available_1k = 29582964 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 169988 cpu_share = 0.57 mem_res = 107740 >>kafka : << mem_virt = 761680 cpu_share = 0.02 mem_res = 672628 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42344 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15398352 free = 9391104 buffers = 996124 cached = 6879540 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 5.19 five_min_avg = 3.96 fifteen_min_avg = 4.05 cpu_share = 5.0 node_type = analytics-alarm-node >> >> 06/02/2025 02:26:12.426 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:12.426 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:12.427 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:12.427 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:12.427 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:12.427 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:12.427 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:12.428 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:26:12.428 7f3bf66d2cc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:14.391 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:26:16.428 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:21.430 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:21.430 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:21.430 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:21.430 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:21.430 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:21.430 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:21.431 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:21.431 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:21.431 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:21.431 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:21.432 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:21.432 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:26:21.433 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:24.693 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:26:25.431 7f3bfaaacb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:30.435 7f3bfaaacb70 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:30.436 7f3bfaaacb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:30.436 7f3bfaaacb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:30.436 7f3bfaaacb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:30.436 7f3bfaaacb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:30.436 7f3bfaaacb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:30.436 7f3bfaaacb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:30.437 7f3bfaaacb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:30.437 7f3bfaaacb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:30.437 7f3bfaaacb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:30.437 7f3bfaaacb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:30.437 7f3bfaaacb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:26:30.437 7f3bfaaacb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:34.436 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:35.225 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:26:39.442 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:39.442 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:39.442 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:39.442 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:39.442 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:39.442 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:39.443 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:39.443 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:39.443 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:39.443 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:39.443 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:39.443 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:26:39.443 7f3bf7cafcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:43.445 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:48.451 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:48.451 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:48.451 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:48.452 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:48.452 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:48.452 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:48.452 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:48.452 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:48.452 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:48.452 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:48.452 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:48.452 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:26:48.453 7f3bf897cc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:52.454 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:57.460 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:57.460 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:57.460 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:57.461 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:57.461 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:57.461 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:57.461 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:57.461 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:57.461 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:57.462 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:57.462 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:57.462 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:26:57.463 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:01.464 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:06.467 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:06.467 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:06.467 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:06.467 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:06.467 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:06.468 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:06.468 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:06.468 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:06.468 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:06.468 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:06.468 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:06.468 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:27:06.469 7f3bf66c9e10 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:10.469 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:10.713 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51411468, partition_space_available_1k=29679016, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168928, cpu_share=0.65, mem_res=107740, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=761728, cpu_share=0.03, mem_res=672632, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42368, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15304920, free=9482708, buffers=996144, cached=6881348, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.46, five_min_avg=3.74, fifteen_min_avg=3.97, cpu_share=2.75, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:27:15.473 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:15.473 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:15.473 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:15.473 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:15.473 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:15.473 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:15.474 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51411468 partition_space_available_1k = 29679016 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 168928 cpu_share = 0.65 mem_res = 107740 >>kafka : << mem_virt = 761728 cpu_share = 0.03 mem_res = 672632 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42368 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15304920 free = 9482708 buffers = 996144 cached = 6881348 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 3.46 five_min_avg = 3.74 fifteen_min_avg = 3.97 cpu_share = 2.75 node_type = analytics-alarm-node >> >> 06/02/2025 02:27:15.474 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:15.474 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:15.474 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:15.474 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:15.474 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:15.475 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:15.475 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:15.475 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:27:15.475 7f3bfacbca20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:19.473 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:24.481 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:24.481 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:24.481 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:24.481 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:24.481 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:24.481 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:24.482 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:24.482 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:24.482 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:24.482 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:24.482 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:24.482 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:27:24.482 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:28.484 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:33.490 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:33.490 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:33.490 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:33.490 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:33.490 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:33.490 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:33.491 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:33.491 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:33.491 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:33.491 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:33.491 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:33.491 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:27:33.491 7f3bf7278978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:37.493 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:39.521 7f3bf89a40f0 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:27:42.497 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:42.497 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:42.498 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:42.498 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:42.498 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:42.498 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:42.498 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:42.498 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:42.498 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:42.498 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:42.499 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:42.499 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:27:42.499 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:46.501 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:51.503 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:51.503 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:51.503 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:51.503 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:51.503 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:51.503 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:51.504 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:51.505 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:51.505 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:51.505 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:51.505 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:51.505 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:27:51.505 7f3bf7caba20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:55.505 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:00.510 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:00.510 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:00.510 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:00.510 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:00.510 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:00.510 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:00.511 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:00.511 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:00.511 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:00.511 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:00.511 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:00.511 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:28:00.511 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:04.514 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:09.516 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:09.516 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:09.516 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:09.516 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:09.516 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:09.516 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:09.518 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:09.518 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:09.518 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:09.518 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:09.518 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:09.518 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:28:09.518 7f3bf66e6f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:10.715 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51413432, partition_space_available_1k=29677052, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168056, cpu_share=0.73, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=761856, cpu_share=0.02, mem_res=672640, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42364, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15299908, free=9484524, buffers=996164, cached=6884524, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.25, five_min_avg=3.63, fifteen_min_avg=3.92, cpu_share=2.62, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:28:10.715 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:10.716 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51413432 partition_space_available_1k = 29677052 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 168056 cpu_share = 0.73 mem_res = 107744 >>kafka : << mem_virt = 761856 cpu_share = 0.02 mem_res = 672640 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42364 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15299908 free = 9484524 buffers = 996164 cached = 6884524 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 3.25 five_min_avg = 3.63 fifteen_min_avg = 3.92 cpu_share = 2.62 node_type = analytics-alarm-node >> >> 06/02/2025 02:28:10.716 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:13.519 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:18.525 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:18.525 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:18.525 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:18.526 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:18.526 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:18.526 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:18.526 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:18.526 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:18.527 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:18.527 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:18.527 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:18.527 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:28:18.527 7f3bf66fd588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:22.527 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:27.534 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:27.534 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:27.534 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:27.534 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:27.534 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:27.534 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:27.535 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:27.535 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:27.535 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:27.535 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:27.535 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:27.535 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:28:27.535 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:31.537 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:36.540 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:36.540 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:36.540 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:36.540 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:36.541 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:36.541 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:36.541 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:36.541 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:36.541 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:36.541 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:36.542 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:36.542 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:28:36.542 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:40.545 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:45.551 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:45.551 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:45.551 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:45.551 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:45.551 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:45.551 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:45.552 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:45.552 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:45.552 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:45.552 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:45.552 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:45.552 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:28:45.552 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:49.555 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:54.560 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:54.561 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:54.561 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:54.561 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:54.561 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:54.561 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:54.561 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:54.562 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:54.562 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:54.562 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:54.562 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:54.562 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:28:54.562 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:58.565 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:03.570 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:03.571 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:03.571 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:03.571 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:03.571 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:03.571 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:03.571 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:03.571 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:03.571 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:03.571 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:03.572 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:03.572 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:29:03.572 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:07.571 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:10.716 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51413980, partition_space_available_1k=29676504, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167016, cpu_share=0.77, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=767976, cpu_share=0.02, mem_res=678756, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42416, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15290572, free=9492416, buffers=996172, cached=6885960, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.53, five_min_avg=3.33, fifteen_min_avg=3.79, cpu_share=2.62, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:29:12.575 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:12.576 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:12.576 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:12.576 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:12.576 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:12.576 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:12.577 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51413980 partition_space_available_1k = 29676504 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 167016 cpu_share = 0.77 mem_res = 107744 >>kafka : << mem_virt = 767976 cpu_share = 0.02 mem_res = 678756 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42416 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15290572 free = 9492416 buffers = 996172 cached = 6885960 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.53 five_min_avg = 3.33 fifteen_min_avg = 3.79 cpu_share = 2.62 node_type = analytics-alarm-node >> >> 06/02/2025 02:29:12.577 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:12.577 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:12.577 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:12.577 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:12.577 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:12.577 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:12.577 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:12.578 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:29:12.578 7f3bf7cd24e0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:16.579 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:21.581 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:21.581 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:21.581 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:21.581 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:21.581 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:21.581 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:21.582 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:21.582 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:21.582 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:21.582 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:21.582 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:21.583 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:29:21.583 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:25.585 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:30.590 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:30.590 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:30.590 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:30.590 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:30.590 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:30.590 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:30.591 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:30.591 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:30.591 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:30.591 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:30.591 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:30.591 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:29:30.591 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:34.594 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:39.600 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:39.600 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:39.600 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:39.600 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:39.600 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:39.600 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:39.601 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:39.601 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:39.601 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:39.601 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:39.601 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:39.601 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:29:39.601 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:43.605 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:48.609 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:48.609 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:48.609 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:48.609 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:48.609 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:48.609 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:48.609 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:48.610 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:48.610 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:48.610 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:48.610 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:48.610 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:29:48.610 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:52.609 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:57.616 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:57.616 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:57.616 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:57.616 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:57.616 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:57.616 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:57.616 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:57.617 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:57.617 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:57.617 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:57.617 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:57.617 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:29:57.617 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:01.620 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:06.624 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:06.625 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:06.625 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:06.625 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:06.625 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:06.626 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:06.627 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:06.627 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:06.627 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:06.627 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:06.627 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:06.628 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:30:06.628 7f3bf66f8390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:10.625 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:10.717 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51419400, partition_space_available_1k=29671084, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=170896, cpu_share=0.69, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=767860, cpu_share=0.02, mem_res=678756, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42440, cpu_share=0.01, mem_res=39616, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15294772, free=9466044, buffers=997756, cached=6906548, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.12, five_min_avg=3.07, fifteen_min_avg=3.67, cpu_share=2.62, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:30:15.632 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:15.632 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:15.632 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:15.632 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:15.632 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:15.632 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:15.633 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51419400 partition_space_available_1k = 29671084 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 170896 cpu_share = 0.69 mem_res = 107744 >>kafka : << mem_virt = 767860 cpu_share = 0.02 mem_res = 678756 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42440 cpu_share = 0.01 mem_res = 39616 >> } system_mem_usage = << total = 32665120 used = 15294772 free = 9466044 buffers = 997756 cached = 6906548 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.12 five_min_avg = 3.07 fifteen_min_avg = 3.67 cpu_share = 2.62 node_type = analytics-alarm-node >> >> 06/02/2025 02:30:15.633 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:15.633 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:15.634 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:15.634 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:15.634 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:15.634 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:15.634 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:15.634 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:30:15.634 7f3bf7cd5828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:19.636 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:24.641 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:24.641 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:24.641 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:24.641 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:24.641 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:24.641 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:24.642 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:24.642 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:24.642 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:24.642 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:24.642 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:24.642 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:30:24.642 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:28.646 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:33.653 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:33.653 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:33.653 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:33.653 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:33.653 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:33.653 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:33.654 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:33.654 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:33.654 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:33.654 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:33.654 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:33.654 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:30:33.654 7f3bf89b1eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:37.655 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:42.667 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:42.667 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:42.667 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:42.667 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:42.667 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:42.667 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:42.668 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:42.668 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:42.668 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:42.668 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:42.668 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:42.668 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:30:42.668 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:46.669 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:51.671 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:51.671 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:51.671 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:51.672 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:51.672 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:51.672 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:51.672 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:51.672 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:51.672 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:51.672 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:51.672 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:51.673 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:30:51.673 7f3bf7108198 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:55.676 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:58.170 7f3bf7cd5780 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:31:00.681 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:00.681 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:00.681 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:00.681 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:00.681 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:00.682 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:00.682 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:00.682 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:00.682 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:00.683 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:00.683 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:00.683 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:31:00.683 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:04.684 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:08.577 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:31:09.687 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:09.688 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:09.688 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:09.688 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:09.688 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:09.688 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:09.689 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:09.689 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:09.689 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:09.689 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:09.689 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:09.689 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:31:09.689 7f3bf7caf4e0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:10.726 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51432304, partition_space_available_1k=29658180, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=169788, cpu_share=0.6, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=767856, cpu_share=0.02, mem_res=678756, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42468, cpu_share=0.01, mem_res=39652, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15349232, free=9387344, buffers=997792, cached=6930752, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.49, five_min_avg=3.96, fifteen_min_avg=3.95, cpu_share=5.0, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:31:10.727 7f3bf66c9438 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:10.727 7f3bf66c9438 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51432304 partition_space_available_1k = 29658180 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 169788 cpu_share = 0.6 mem_res = 107744 >>kafka : << mem_virt = 767856 cpu_share = 0.02 mem_res = 678756 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42468 cpu_share = 0.01 mem_res = 39652 >> } system_mem_usage = << total = 32665120 used = 15349232 free = 9387344 buffers = 997792 cached = 6930752 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 5.49 five_min_avg = 3.96 fifteen_min_avg = 3.95 cpu_share = 5.0 node_type = analytics-alarm-node >> >> 06/02/2025 02:31:10.727 7f3bf66c9438 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:13.690 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:18.697 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:18.697 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:18.698 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:18.698 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:18.698 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:18.698 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:18.698 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:18.699 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:18.699 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:18.699 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:18.699 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:18.699 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:31:18.699 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:19.158 7f3bf7278e10 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:31:22.698 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:27.705 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:27.705 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:27.705 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:27.706 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:27.706 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:27.706 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:27.706 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:27.706 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:27.706 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:27.706 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:27.706 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:27.706 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:31:27.706 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:31.709 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:36.710 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:36.711 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:36.711 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:36.711 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:36.711 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:36.711 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:36.712 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:36.712 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:36.712 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:36.712 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:36.712 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:36.712 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:31:36.712 7f3bf7cd5588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:40.713 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:45.719 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:45.719 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:45.719 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:45.719 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:45.719 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:45.719 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:45.720 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:45.720 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:45.720 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:45.720 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:45.720 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:45.720 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:31:45.720 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:49.722 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:54.727 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:54.727 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:54.727 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:54.728 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:54.728 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:54.729 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:54.729 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:54.729 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:54.730 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:54.730 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:54.730 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:54.731 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:31:54.731 7f3bf7cd2b70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:58.731 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:03.737 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:03.737 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:03.737 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:03.737 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:03.737 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:03.737 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:03.738 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:03.738 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:03.738 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:03.738 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:03.738 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:03.738 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:32:03.738 7f3bf7278630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:07.738 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:10.719 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51431108, partition_space_available_1k=29659376, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168772, cpu_share=0.67, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=775092, cpu_share=0.03, mem_res=685976, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42528, cpu_share=0.01, mem_res=39652, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15322100, free=9411000, buffers=997792, cached=6934228, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.87, five_min_avg=3.49, fifteen_min_avg=3.78, cpu_share=2.5, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:32:12.741 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:12.741 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:12.741 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:12.742 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:12.742 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:12.742 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:12.742 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51431108 partition_space_available_1k = 29659376 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 168772 cpu_share = 0.67 mem_res = 107744 >>kafka : << mem_virt = 775092 cpu_share = 0.03 mem_res = 685976 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42528 cpu_share = 0.01 mem_res = 39652 >> } system_mem_usage = << total = 32665120 used = 15322100 free = 9411000 buffers = 997792 cached = 6934228 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.87 five_min_avg = 3.49 fifteen_min_avg = 3.78 cpu_share = 2.5 node_type = analytics-alarm-node >> >> 06/02/2025 02:32:12.742 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:12.742 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:12.743 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:12.743 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:12.743 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:12.744 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:12.744 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:12.744 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:32:12.744 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:16.745 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:21.751 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:21.751 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:21.751 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:21.752 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:21.752 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:21.752 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:21.753 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:21.753 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:21.753 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:21.753 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:21.753 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:21.753 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:32:21.753 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:25.755 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:30.761 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:30.762 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:30.762 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:30.762 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:30.762 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:30.762 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:30.764 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:30.764 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:30.764 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:30.764 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:30.764 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:30.764 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:32:30.764 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:34.765 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:36.115 7f3bf7cad390 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:32:39.769 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:39.770 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:39.770 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:39.770 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:39.770 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:39.770 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:39.771 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:39.771 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:39.771 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:39.771 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:39.771 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:39.771 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:32:39.771 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:43.773 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:48.778 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:48.778 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:48.778 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:48.778 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:48.778 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:48.778 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:48.779 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:48.779 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:48.779 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:48.779 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:48.779 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:48.779 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:32:48.779 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:52.779 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:57.784 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:57.784 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:57.784 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:57.785 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:57.785 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:57.785 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:57.785 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:57.785 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:57.786 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:57.786 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:57.786 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:57.786 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:32:57.786 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:01.788 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:06.791 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:06.791 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:06.791 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:06.792 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:06.792 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:06.792 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:06.793 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:06.793 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:06.793 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:06.793 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:06.793 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:06.793 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:33:06.793 7f3bf84bf828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:10.726 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51432860, partition_space_available_1k=29657624, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167816, cpu_share=0.8, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=779684, cpu_share=0.03, mem_res=690540, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42520, cpu_share=0.01, mem_res=39652, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15309668, free=9420872, buffers=997792, cached=6936788, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.38, five_min_avg=3.26, fifteen_min_avg=3.69, cpu_share=2.75, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:33:10.726 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:10.726 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51432860 partition_space_available_1k = 29657624 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 167816 cpu_share = 0.8 mem_res = 107744 >>kafka : << mem_virt = 779684 cpu_share = 0.03 mem_res = 690540 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42520 cpu_share = 0.01 mem_res = 39652 >> } system_mem_usage = << total = 32665120 used = 15309668 free = 9420872 buffers = 997792 cached = 6936788 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.38 five_min_avg = 3.26 fifteen_min_avg = 3.69 cpu_share = 2.75 node_type = analytics-alarm-node >> >> 06/02/2025 02:33:10.726 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:10.791 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:15.797 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:15.797 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:15.798 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:15.798 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:15.798 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:15.798 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:15.798 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:15.798 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:15.799 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:15.799 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:15.799 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:15.799 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:33:15.799 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:19.801 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:24.805 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:24.805 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:24.806 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:24.806 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:24.806 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:24.806 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:24.806 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:24.806 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:24.807 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:24.807 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:24.807 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:24.807 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:33:24.807 7f3bf89a4630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:28.808 7f3bf7cd1978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:33.815 7f3bf7cd1978 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:33.815 7f3bf7cd1978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:33.815 7f3bf7cd1978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:33.815 7f3bf7cd1978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:33.816 7f3bf7cd1978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:33.816 7f3bf7cd1978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:33.816 7f3bf7cd1978 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:33.816 7f3bf7cd1978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:33.816 7f3bf7cd1978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:33.816 7f3bf7cd1978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:33.816 7f3bf7cd1978 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:33.817 7f3bf7cd1978 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:33:33.817 7f3bf7cd1978 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:37.815 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:42.821 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:42.821 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:42.821 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:42.821 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:42.821 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:42.821 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:42.822 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:42.822 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:42.822 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:42.822 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:42.822 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:42.822 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:33:42.822 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:46.825 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:51.831 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:51.832 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:51.832 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:51.832 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:51.832 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:51.832 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:51.833 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:51.833 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:51.833 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:51.833 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:51.834 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:51.834 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:33:51.834 7f3bf89c3048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:55.834 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:00.839 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:00.840 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:00.840 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:00.840 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:00.840 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:00.840 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:00.840 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:00.840 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:00.841 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:00.841 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:00.841 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:00.841 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:34:00.841 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:04.843 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:09.846 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:09.847 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:09.847 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:09.847 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:09.847 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:09.847 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:09.848 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:09.848 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:09.848 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:09.848 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:09.848 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:09.848 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:34:09.849 7f3bf7cbef60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:10.727 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51433700, partition_space_available_1k=29656784, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166836, cpu_share=0.82, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=774268, cpu_share=0.02, mem_res=685136, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42544, cpu_share=0.01, mem_res=39652, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15307912, free=9420964, buffers=997792, cached=6938452, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.08, five_min_avg=3.02, fifteen_min_avg=3.58, cpu_share=2.62, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:34:10.728 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:10.728 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51433700 partition_space_available_1k = 29656784 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 166836 cpu_share = 0.82 mem_res = 107744 >>kafka : << mem_virt = 774268 cpu_share = 0.02 mem_res = 685136 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42544 cpu_share = 0.01 mem_res = 39652 >> } system_mem_usage = << total = 32665120 used = 15307912 free = 9420964 buffers = 997792 cached = 6938452 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.08 five_min_avg = 3.02 fifteen_min_avg = 3.58 cpu_share = 2.62 node_type = analytics-alarm-node >> >> 06/02/2025 02:34:10.728 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:13.849 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:18.856 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:18.856 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:18.856 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:18.856 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:18.857 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:18.857 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:18.857 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:18.857 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:18.857 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:18.857 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:18.857 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:18.858 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:34:18.858 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:22.860 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:27.866 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:27.866 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:27.866 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:27.867 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:27.867 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:27.867 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:27.867 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:27.867 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:27.867 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:27.868 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:27.868 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:27.868 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:34:27.868 7f3bf7cafb70 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:31.871 7f3bf66e6ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:36.877 7f3bf66e6ac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:36.878 7f3bf66e6ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:36.878 7f3bf66e6ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:36.878 7f3bf66e6ac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:36.878 7f3bf66e6ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:36.878 7f3bf66e6ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:36.878 7f3bf66e6ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:36.879 7f3bf66e6ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:36.879 7f3bf66e6ac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:36.879 7f3bf66e6ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:36.879 7f3bf66e6ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:36.879 7f3bf66e6ac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:34:36.879 7f3bf66e6ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:40.880 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:45.886 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:45.886 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:45.887 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:45.887 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:45.887 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:45.887 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:45.887 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:45.887 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:45.888 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:45.888 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:45.888 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:45.888 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:34:45.888 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:49.890 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:54.893 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:54.893 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:54.893 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:54.893 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:54.893 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:54.893 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:54.894 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:54.894 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:54.894 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:54.894 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:54.895 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:54.895 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:34:54.895 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:58.897 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:03.904 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:03.904 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:03.904 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:03.904 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:03.904 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:03.904 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:03.905 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:03.905 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:03.905 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:03.905 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:03.905 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:03.905 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:35:03.905 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:07.905 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:10.729 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51439272, partition_space_available_1k=29651212, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=170684, cpu_share=0.85, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=774264, cpu_share=0.02, mem_res=685136, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42596, cpu_share=0.01, mem_res=39652, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15314792, free=9407832, buffers=997912, cached=6944584, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.79, five_min_avg=2.76, fifteen_min_avg=3.45, cpu_share=2.75, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:35:12.909 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:12.909 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:12.909 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:12.909 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:12.909 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:12.910 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:12.910 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51439272 partition_space_available_1k = 29651212 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 170684 cpu_share = 0.85 mem_res = 107744 >>kafka : << mem_virt = 774264 cpu_share = 0.02 mem_res = 685136 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42596 cpu_share = 0.01 mem_res = 39652 >> } system_mem_usage = << total = 32665120 used = 15314792 free = 9407832 buffers = 997912 cached = 6944584 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 1.79 five_min_avg = 2.76 fifteen_min_avg = 3.45 cpu_share = 2.75 node_type = analytics-alarm-node >> >> 06/02/2025 02:35:12.910 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:12.910 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:12.910 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:12.910 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:12.910 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:12.911 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:12.911 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:12.911 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:35:12.911 7f3bf7cada20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:16.913 7f3bf725be10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:21.920 7f3bf725be10 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:21.920 7f3bf725be10 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:21.920 7f3bf725be10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:21.921 7f3bf725be10 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:21.921 7f3bf725be10 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:21.921 7f3bf725be10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:21.921 7f3bf725be10 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:21.921 7f3bf725be10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:21.921 7f3bf725be10 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:21.921 7f3bf725be10 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:21.921 7f3bf725be10 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:21.921 7f3bf725be10 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:35:21.922 7f3bf725be10 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:25.923 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:30.929 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:30.929 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:30.929 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:30.930 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:30.930 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:30.930 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:30.930 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:30.930 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:30.931 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:30.931 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:30.931 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:30.931 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:35:30.931 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:34.933 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:39.936 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:39.936 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:39.936 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:39.937 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:39.937 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:39.937 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:39.937 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:39.937 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:39.938 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:39.938 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:39.938 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:39.938 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:35:39.938 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:43.936 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:48.941 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:48.941 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:48.941 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:48.941 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:48.941 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:48.941 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:48.942 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:48.942 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:48.942 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:48.942 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:48.942 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:48.942 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:35:48.942 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:52.945 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:57.951 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:57.951 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:57.951 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:57.951 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:57.951 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:57.951 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:57.952 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:57.952 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:57.952 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:57.952 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:57.952 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:57.952 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:35:57.952 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:01.951 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:06.954 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:06.954 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:06.955 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:06.955 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:06.955 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:06.955 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:06.956 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:06.956 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:06.956 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:06.956 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:06.956 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:06.956 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:36:06.956 7f3bfbeacac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:10.744 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51440008, partition_space_available_1k=29650476, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=169700, cpu_share=0.9, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=790160, cpu_share=0.03, mem_res=700984, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42576, cpu_share=0.01, mem_res=39636, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15401080, free=9314604, buffers=997976, cached=6951460, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.92, five_min_avg=2.63, fifteen_min_avg=3.36, cpu_share=3.0, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:36:10.744 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:10.744 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51440008 partition_space_available_1k = 29650476 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 169700 cpu_share = 0.9 mem_res = 107744 >>kafka : << mem_virt = 790160 cpu_share = 0.03 mem_res = 700984 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42576 cpu_share = 0.01 mem_res = 39636 >> } system_mem_usage = << total = 32665120 used = 15401080 free = 9314604 buffers = 997976 cached = 6951460 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 1.92 five_min_avg = 2.63 fifteen_min_avg = 3.36 cpu_share = 3.0 node_type = analytics-alarm-node >> >> 06/02/2025 02:36:10.744 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:10.955 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:14.453 7f3bf7cbe630 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:36:15.959 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:15.959 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:15.959 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:15.959 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:15.960 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:15.960 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:15.960 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:15.961 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:15.961 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:15.961 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:15.961 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:15.961 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:36:15.961 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:19.962 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:24.966 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:24.966 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:24.966 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:24.966 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:24.966 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:24.966 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:24.970 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:24.971 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:24.971 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:24.971 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:24.971 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:24.973 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:36:24.973 7f3bf66d2828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:24.988 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:36:28.969 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:33.975 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:33.975 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:33.975 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:33.976 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:33.976 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:33.976 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:33.976 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:33.976 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:33.976 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:33.976 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:33.976 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:33.977 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:36:33.977 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:37.978 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:40.414 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:36:42.981 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:42.981 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:42.981 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:42.981 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:42.981 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:42.981 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:42.982 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:42.982 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:42.982 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:42.982 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:42.982 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:42.982 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:36:42.982 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:46.985 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:51.991 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:51.991 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:51.991 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:51.992 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:51.992 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:51.992 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:51.992 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:51.993 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:51.993 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:51.993 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:51.993 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:51.993 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:36:51.993 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:55.993 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:00.999 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:01.000 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:01.001 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:01.001 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:01.001 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:01.001 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:01.002 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:01.002 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:01.002 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:01.002 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:01.002 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:01.002 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:37:01.002 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:05.003 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:10.005 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:10.006 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:10.006 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:10.006 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:10.006 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:10.006 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:10.007 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:10.007 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:10.007 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:10.007 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:10.007 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:10.007 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:37:10.007 7f3bf66ff6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:10.731 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51450860, partition_space_available_1k=29639624, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168588, cpu_share=0.67, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=790168, cpu_share=0.02, mem_res=701004, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42624, cpu_share=0.01, mem_res=39660, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15351964, free=9348068, buffers=997988, cached=6967100, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.21, five_min_avg=3.57, fifteen_min_avg=3.65, cpu_share=5.25, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:37:10.731 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:10.731 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51450860 partition_space_available_1k = 29639624 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 168588 cpu_share = 0.67 mem_res = 107744 >>kafka : << mem_virt = 790168 cpu_share = 0.02 mem_res = 701004 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42624 cpu_share = 0.01 mem_res = 39660 >> } system_mem_usage = << total = 32665120 used = 15351964 free = 9348068 buffers = 997988 cached = 6967100 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 5.21 five_min_avg = 3.57 fifteen_min_avg = 3.65 cpu_share = 5.25 node_type = analytics-alarm-node >> >> 06/02/2025 02:37:10.731 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:14.009 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:19.013 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:19.013 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:19.013 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:19.013 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:19.013 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:19.013 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:19.014 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:19.014 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:19.014 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:19.014 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:19.014 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:19.014 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:37:19.014 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:23.018 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:28.022 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:28.022 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:28.022 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:28.022 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:28.023 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:28.023 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:28.023 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:28.023 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:28.023 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:28.024 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:28.024 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:28.024 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:37:28.024 7f3bf7cad828 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:32.026 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:37.032 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:37.032 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:37.032 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:37.032 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:37.033 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:37.033 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:37.033 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:37.033 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:37.033 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:37.034 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:37.034 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:37.034 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:37:37.034 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:41.033 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:46.038 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:46.038 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:46.038 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:46.038 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:46.039 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:46.039 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:46.039 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:46.039 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:46.039 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:46.039 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:46.039 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:46.039 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:37:46.039 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:50.042 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:55.049 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:55.049 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:55.049 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:55.049 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:55.049 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:55.049 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:55.050 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:55.050 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:55.050 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:55.050 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:55.050 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:55.050 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:37:55.050 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:59.050 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:04.056 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:04.056 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:04.056 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:04.056 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:04.056 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:04.056 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:04.057 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:04.057 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:04.057 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:04.057 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:04.057 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:04.057 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:38:04.058 7f3bf7264a20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:08.058 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:10.735 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51453108, partition_space_available_1k=29637376, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167572, cpu_share=0.65, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=790308, cpu_share=0.03, mem_res=701012, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42676, cpu_share=0.01, mem_res=39660, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15360104, free=9337036, buffers=997988, cached=6969992, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.78, five_min_avg=3.44, fifteen_min_avg=3.6, cpu_share=2.5, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:38:13.063 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:13.063 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:13.064 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:13.064 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:13.064 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:13.064 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:13.064 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51453108 partition_space_available_1k = 29637376 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 167572 cpu_share = 0.65 mem_res = 107744 >>kafka : << mem_virt = 790308 cpu_share = 0.03 mem_res = 701012 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42676 cpu_share = 0.01 mem_res = 39660 >> } system_mem_usage = << total = 32665120 used = 15360104 free = 9337036 buffers = 997988 cached = 6969992 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 3.78 five_min_avg = 3.44 fifteen_min_avg = 3.6 cpu_share = 2.5 node_type = analytics-alarm-node >> >> 06/02/2025 02:38:13.064 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:13.064 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:13.065 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:13.065 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:13.065 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:13.065 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:13.065 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:13.065 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:38:13.065 7f3bf7cd2ac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:17.067 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:22.074 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:22.075 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:22.075 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:22.075 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:22.075 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:22.075 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:22.075 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:22.075 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:22.076 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:22.076 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:22.076 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:22.076 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:38:22.076 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:24.506 7f3bfa9bf0f0 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:38:26.074 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:31.081 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:31.081 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:31.082 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:31.082 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:31.082 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:31.082 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:31.083 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:31.083 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:31.083 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:31.083 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:31.083 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:31.083 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:38:31.083 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:35.085 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:40.091 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:40.091 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:40.091 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:40.091 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:40.091 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:40.091 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:40.092 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:40.092 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:40.092 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:40.092 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:40.092 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:40.092 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:38:40.092 7f3bfaab1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:44.093 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:49.100 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:49.100 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:49.100 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:49.100 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:49.100 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:49.100 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:49.101 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:49.101 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:49.101 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:49.101 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:49.101 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:49.101 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:38:49.101 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:53.102 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:58.106 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:58.107 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:58.107 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:58.107 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:58.107 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:58.107 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:58.108 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:58.108 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:58.108 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:58.108 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:58.108 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:58.108 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:38:58.108 7f3bf66fd2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:02.111 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:07.114 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:07.114 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:07.115 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:07.115 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:07.115 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:07.115 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:07.116 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:07.116 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:07.116 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:07.116 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:07.116 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:07.116 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:39:07.116 7f3bf725fcc0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:10.736 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51454244, partition_space_available_1k=29636240, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166728, cpu_share=0.68, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=790192, cpu_share=0.02, mem_res=701032, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42672, cpu_share=0.01, mem_res=39660, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15344976, free=9349992, buffers=997996, cached=6972156, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.81, five_min_avg=3.23, fifteen_min_avg=3.51, cpu_share=2.62, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:39:10.736 7f3bf7cab630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:10.737 7f3bf7cab630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51454244 partition_space_available_1k = 29636240 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 166728 cpu_share = 0.68 mem_res = 107744 >>kafka : << mem_virt = 790192 cpu_share = 0.02 mem_res = 701032 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42672 cpu_share = 0.01 mem_res = 39660 >> } system_mem_usage = << total = 32665120 used = 15344976 free = 9349992 buffers = 997996 cached = 6972156 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.81 five_min_avg = 3.23 fifteen_min_avg = 3.51 cpu_share = 2.62 node_type = analytics-alarm-node >> >> 06/02/2025 02:39:10.737 7f3bf7cab630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:11.115 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:16.121 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:16.122 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:16.122 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:16.122 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:16.122 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:16.123 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:16.123 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:16.123 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:16.123 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:16.123 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:16.123 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:16.124 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:39:16.124 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:20.124 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:25.131 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:25.131 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:25.131 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:25.131 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:25.131 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:25.131 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:25.132 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:25.132 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:25.132 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:25.132 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:25.132 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:25.132 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:39:25.132 7f3bfac99390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:29.134 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:34.137 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:34.137 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:34.137 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:34.137 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:34.137 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:34.137 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:34.138 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:34.138 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:34.138 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:34.138 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:34.138 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:34.138 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:39:34.138 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:38.141 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:43.145 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:43.145 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:43.145 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:43.145 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:43.145 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:43.145 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:43.146 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:43.146 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:43.146 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:43.146 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:43.146 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:43.146 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:39:43.146 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:47.149 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:52.153 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:52.153 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:52.153 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:52.153 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:52.153 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:52.154 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:52.154 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:52.154 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:52.154 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:52.155 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:52.155 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:52.155 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:39:52.155 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:56.157 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:01.164 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:01.165 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:01.165 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:01.166 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:01.166 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:01.166 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:01.167 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:01.167 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:01.167 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:01.167 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:01.167 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:01.168 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:40:01.168 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:05.168 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:10.171 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:10.172 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:10.172 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:10.172 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:10.172 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:10.172 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:10.173 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:10.173 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:10.173 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:10.173 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:10.173 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:10.173 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:40:10.173 7f3bfaac90f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:10.742 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51459704, partition_space_available_1k=29630780, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=170512, cpu_share=0.72, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=820516, cpu_share=0.03, mem_res=731288, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42696, cpu_share=0.01, mem_res=39660, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15426152, free=9257788, buffers=998000, cached=6983180, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.52, five_min_avg=3.04, fifteen_min_avg=3.43, cpu_share=2.62, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:40:10.742 7f3bf7ca4d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:10.743 7f3bf7ca4d68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51459704 partition_space_available_1k = 29630780 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 170512 cpu_share = 0.72 mem_res = 107744 >>kafka : << mem_virt = 820516 cpu_share = 0.03 mem_res = 731288 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42696 cpu_share = 0.01 mem_res = 39660 >> } system_mem_usage = << total = 32665120 used = 15426152 free = 9257788 buffers = 998000 cached = 6983180 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.52 five_min_avg = 3.04 fifteen_min_avg = 3.43 cpu_share = 2.62 node_type = analytics-alarm-node >> >> 06/02/2025 02:40:10.743 7f3bf7ca4d68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:14.174 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:17.777 7f3c00c9a048 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:40:19.177 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:19.177 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:19.177 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:19.177 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:19.177 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:19.177 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:19.178 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:19.178 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:19.178 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:19.178 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:19.178 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:19.178 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:40:19.178 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:23.180 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:28.182 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:28.182 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:28.182 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:28.182 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:28.182 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:28.182 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:28.183 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:28.183 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:28.183 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:28.183 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:28.183 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:28.183 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:40:28.183 7f3bf7cabc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:28.213 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:40:32.185 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:37.191 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:37.191 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:37.191 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:37.191 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:37.192 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:37.192 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:37.192 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:37.192 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:37.192 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:37.192 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:37.192 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:37.192 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:40:37.193 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:40.442 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:40:41.191 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:46.201 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:46.201 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:46.201 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:46.201 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:46.201 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:46.201 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:46.202 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:46.202 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:46.202 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:46.202 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:46.202 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:46.202 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:40:46.202 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:50.206 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:55.212 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:55.212 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:55.212 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:55.213 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:55.213 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:55.213 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:55.213 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:55.214 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:55.214 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:55.214 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:55.214 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:55.214 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:40:55.214 7f3bf84af780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:59.214 7f3bf7ca4780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:04.219 7f3bf7ca4780 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:04.219 7f3bf7ca4780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:04.219 7f3bf7ca4780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:04.220 7f3bf7ca4780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:04.220 7f3bf7ca4780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:04.220 7f3bf7ca4780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:04.221 7f3bf7ca4780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:04.221 7f3bf7ca4780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:04.221 7f3bf7ca4780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:04.221 7f3bf7ca4780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:04.221 7f3bf7ca4780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:04.221 7f3bf7ca4780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:41:04.221 7f3bf7ca4780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:08.221 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:10.754 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51470336, partition_space_available_1k=29620148, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=169576, cpu_share=0.62, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=846048, cpu_share=0.02, mem_res=756732, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42752, cpu_share=0.01, mem_res=39660, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15425168, free=9239848, buffers=998008, cached=7002096, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.89, five_min_avg=4.37, fifteen_min_avg=3.87, cpu_share=5.12, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:41:13.224 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:13.224 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:13.224 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:13.225 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:13.225 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:13.225 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:13.225 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51470336 partition_space_available_1k = 29620148 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 169576 cpu_share = 0.62 mem_res = 107744 >>kafka : << mem_virt = 846048 cpu_share = 0.02 mem_res = 756732 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42752 cpu_share = 0.01 mem_res = 39660 >> } system_mem_usage = << total = 32665120 used = 15425168 free = 9239848 buffers = 998008 cached = 7002096 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 6.89 five_min_avg = 4.37 fifteen_min_avg = 3.87 cpu_share = 5.12 node_type = analytics-alarm-node >> >> 06/02/2025 02:41:13.225 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:13.225 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:13.226 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:13.226 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:13.226 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:13.226 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:13.226 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:13.226 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:41:13.226 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:17.225 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:22.229 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:22.230 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:22.230 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:22.230 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:22.230 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:22.230 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:22.230 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:22.230 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:22.231 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:22.231 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:22.231 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:22.231 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:41:22.231 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:26.234 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:31.239 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:31.240 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:31.240 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:31.240 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:31.240 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:31.240 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:31.241 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:31.241 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:31.241 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:31.241 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:31.241 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:31.241 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:41:31.241 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:35.244 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:40.249 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:40.249 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:40.249 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:40.249 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:40.249 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:40.249 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:40.250 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:40.250 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:40.250 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:40.250 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:40.250 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:40.250 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:41:40.250 7f3bfa9112e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:44.251 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:49.257 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:49.257 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:49.257 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:49.258 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:49.258 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:49.258 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:49.258 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:49.258 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:49.258 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:49.258 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:49.258 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:49.258 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:41:49.258 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:53.261 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:58.264 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:58.264 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:58.265 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:58.265 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:58.265 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:58.265 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:58.265 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:58.265 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:58.265 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:58.265 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:58.266 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:58.266 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:41:58.266 7f3bf7cc4048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:02.268 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:07.270 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:07.271 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:07.271 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:07.271 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:07.271 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:07.271 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:07.272 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:07.272 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:07.272 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:07.272 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:07.272 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:07.272 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:42:07.272 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:10.742 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51472464, partition_space_available_1k=29618020, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168420, cpu_share=0.7, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=846148, cpu_share=0.02, mem_res=756732, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42740, cpu_share=0.01, mem_res=39660, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15421660, free=9240520, buffers=998008, cached=7004932, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.72, five_min_avg=3.92, fifteen_min_avg=3.74, cpu_share=2.5, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:42:10.742 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:10.743 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51472464 partition_space_available_1k = 29618020 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 168420 cpu_share = 0.7 mem_res = 107744 >>kafka : << mem_virt = 846148 cpu_share = 0.02 mem_res = 756732 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42740 cpu_share = 0.01 mem_res = 39660 >> } system_mem_usage = << total = 32665120 used = 15421660 free = 9240520 buffers = 998008 cached = 7004932 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 3.72 five_min_avg = 3.92 fifteen_min_avg = 3.74 cpu_share = 2.5 node_type = analytics-alarm-node >> >> 06/02/2025 02:42:10.743 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:11.272 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:16.277 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:16.277 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:16.277 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:16.278 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:16.278 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:16.278 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:16.278 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:16.278 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:16.278 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:16.279 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:16.279 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:16.279 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:42:16.279 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:20.281 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:25.285 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:25.285 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:25.285 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:25.285 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:25.285 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:25.285 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:25.286 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:25.286 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:25.286 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:25.286 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:25.286 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:25.286 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:42:25.286 7f3c01be30f0 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:29.286 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:34.293 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:34.293 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:34.293 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:34.293 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:34.293 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:34.293 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:34.294 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:34.294 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:34.294 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:34.294 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:34.294 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:34.294 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:42:34.294 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:38.296 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:43.298 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:43.299 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:43.299 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:43.299 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:43.299 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:43.299 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:43.299 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:43.300 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:43.300 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:43.300 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:43.300 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:43.300 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:42:43.300 7f3bf7cd52e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:47.303 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:52.309 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:52.309 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:52.309 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:52.309 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:52.309 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:52.309 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:52.310 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:52.310 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:52.310 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:52.310 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:52.310 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:52.310 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:42:52.310 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:56.313 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:01.318 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:01.318 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:01.318 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:01.318 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:01.319 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:01.319 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:01.319 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:01.319 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:01.319 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:01.319 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:01.319 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:01.320 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:43:01.320 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:05.322 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:10.325 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:10.325 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:10.325 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:10.325 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:10.326 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:10.326 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:10.326 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:10.326 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:10.327 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:10.327 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:10.327 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:10.327 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:43:10.327 7f3bfbeb2630 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:10.751 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51473740, partition_space_available_1k=29616744, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167396, cpu_share=0.75, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=846140, cpu_share=0.02, mem_res=756732, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42772, cpu_share=0.01, mem_res=39660, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15507992, free=9147068, buffers=998008, cached=7012052, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.64, five_min_avg=3.86, fifteen_min_avg=3.74, cpu_share=2.62, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:43:10.751 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:10.751 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51473740 partition_space_available_1k = 29616744 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 167396 cpu_share = 0.75 mem_res = 107744 >>kafka : << mem_virt = 846140 cpu_share = 0.02 mem_res = 756732 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42772 cpu_share = 0.01 mem_res = 39660 >> } system_mem_usage = << total = 32665120 used = 15507992 free = 9147068 buffers = 998008 cached = 7012052 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 3.64 five_min_avg = 3.86 fifteen_min_avg = 3.74 cpu_share = 2.62 node_type = analytics-alarm-node >> >> 06/02/2025 02:43:10.751 7f3bfae05f60 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:14.326 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:16.502 7f3bf66e5d68 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:43:19.331 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:19.331 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:19.331 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:19.331 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:19.331 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:19.331 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:19.332 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:19.332 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:19.332 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:19.332 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:19.332 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:19.332 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:43:19.332 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:23.335 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:28.337 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:28.337 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:28.337 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:28.337 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:28.337 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:28.337 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:28.338 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:28.338 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:28.338 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:28.338 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:28.338 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:28.338 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:43:28.338 7f3bf66ff390 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:32.342 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:37.348 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:37.349 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:37.349 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:37.349 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:37.349 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:37.349 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:37.350 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:37.350 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:37.350 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:37.350 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:37.350 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:37.350 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:43:37.350 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:41.352 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:46.356 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:46.357 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:46.357 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:46.357 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:46.357 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:46.357 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:46.358 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:46.358 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:46.358 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:46.359 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:46.359 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:46.359 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:43:46.359 7f3bf66fdeb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:50.356 7f3bf66d2780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:55.363 7f3bf66d2780 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:55.363 7f3bf66d2780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:55.364 7f3bf66d2780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:55.364 7f3bf66d2780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:55.364 7f3bf66d2780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:55.364 7f3bf66d2780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:55.364 7f3bf66d2780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:55.364 7f3bf66d2780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:55.365 7f3bf66d2780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:55.365 7f3bf66d2780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:55.365 7f3bf66d2780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:55.365 7f3bf66d2780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:43:55.365 7f3bf66d2780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:59.363 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:04.369 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:04.369 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:04.369 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:04.369 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:04.370 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:04.370 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:04.370 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:04.370 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:04.371 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:04.371 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:04.371 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:04.371 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:44:04.371 7f3c00a4c240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:08.370 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:10.745 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51479788, partition_space_available_1k=29610696, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=171516, cpu_share=0.82, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=846136, cpu_share=0.02, mem_res=756732, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42824, cpu_share=0.01, mem_res=39660, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15402276, free=9255088, buffers=998008, cached=7009748, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.77, five_min_avg=3.57, fifteen_min_avg=3.64, cpu_share=2.62, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:44:13.375 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:13.375 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:13.375 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:13.376 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:13.376 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:13.376 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:13.376 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51479788 partition_space_available_1k = 29610696 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 171516 cpu_share = 0.82 mem_res = 107744 >>kafka : << mem_virt = 846136 cpu_share = 0.02 mem_res = 756732 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42824 cpu_share = 0.01 mem_res = 39660 >> } system_mem_usage = << total = 32665120 used = 15402276 free = 9255088 buffers = 998008 cached = 7009748 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 2.77 five_min_avg = 3.57 fifteen_min_avg = 3.64 cpu_share = 2.62 node_type = analytics-alarm-node >> >> 06/02/2025 02:44:13.376 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:13.376 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:13.376 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:13.377 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:13.377 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:13.377 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:13.377 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:13.377 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:44:13.377 7f3bf66e6048 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:17.379 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:22.385 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:22.385 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:22.385 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:22.386 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:22.386 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:22.386 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:22.386 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:22.387 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:22.387 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:22.387 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:22.387 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:22.387 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:44:22.387 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:26.388 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:31.391 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:31.391 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:31.391 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:31.392 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:31.392 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:31.392 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:31.392 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:31.392 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:31.393 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:31.393 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:31.393 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:31.393 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:44:31.393 7f3bf7cab2e8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:35.395 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:40.401 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:40.401 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:40.401 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:40.401 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:40.401 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:40.401 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:40.402 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:40.402 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:40.402 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:40.402 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:40.402 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:40.402 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:44:40.402 7f3bfadc3240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:44.402 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:49.407 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:49.408 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:49.408 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:49.408 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:49.408 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:49.408 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:49.409 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:49.409 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:49.409 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:49.409 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:49.409 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:49.409 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:44:49.409 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:53.411 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:58.417 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:58.417 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:58.418 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:58.418 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:58.418 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:58.418 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:58.419 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:58.419 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:58.419 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:58.419 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:58.419 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:58.419 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:44:58.419 7f3bf7cd1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:02.421 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:07.424 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:07.424 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:07.425 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:07.425 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:07.425 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:07.425 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:07.426 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:07.426 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:07.426 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:07.426 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:07.426 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:07.426 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:45:07.426 7f3bf84af6d8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:10.750 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51480452, partition_space_available_1k=29610032, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=170500, cpu_share=0.89, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=857180, cpu_share=0.02, mem_res=767844, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42820, cpu_share=0.01, mem_res=39660, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15413020, free=9237944, buffers=998008, cached=7016148, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.16, five_min_avg=3.48, fifteen_min_avg=3.6, cpu_share=2.75, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:45:10.750 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:10.750 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51480452 partition_space_available_1k = 29610032 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 170500 cpu_share = 0.89 mem_res = 107744 >>kafka : << mem_virt = 857180 cpu_share = 0.02 mem_res = 767844 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42820 cpu_share = 0.01 mem_res = 39660 >> } system_mem_usage = << total = 32665120 used = 15413020 free = 9237944 buffers = 998008 cached = 7016148 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 3.16 five_min_avg = 3.48 fifteen_min_avg = 3.6 cpu_share = 2.75 node_type = analytics-alarm-node >> >> 06/02/2025 02:45:10.750 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:11.425 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:16.429 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:16.429 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:16.429 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:16.429 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:16.429 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:16.429 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:16.430 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:16.430 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:16.430 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:16.430 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:16.430 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:16.430 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:45:16.430 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:20.434 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:25.442 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:25.443 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:25.443 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:25.443 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:25.443 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:25.443 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:25.443 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:25.444 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:25.444 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:25.444 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:25.444 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:25.444 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:45:25.444 7f3bf7278240 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:29.443 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:34.449 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:34.449 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:34.449 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:34.449 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:34.449 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:34.449 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:34.450 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:34.450 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:34.450 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:34.450 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:34.450 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:34.450 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:45:34.451 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:38.452 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:43.459 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:43.459 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:43.459 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:43.459 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:43.459 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:43.459 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:43.460 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:43.460 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:43.460 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:43.460 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:43.460 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:43.460 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:45:43.460 7f3bf66fdc18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:47.462 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:52.469 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:52.469 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:52.469 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:52.469 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:52.469 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:52.469 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:52.469 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:52.470 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:52.470 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:52.470 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:52.470 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:52.470 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:45:52.470 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:55.610 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:45:56.469 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:01.472 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:01.472 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:01.472 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:01.472 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:01.473 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:01.473 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:01.473 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:01.474 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:01.475 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:01.475 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:01.475 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:01.475 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:46:01.475 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:05.477 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:05.683 7f3bf7cabd68 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:46:10.480 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:10.480 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:10.480 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:10.481 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:10.481 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:10.481 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:10.482 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:10.482 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:10.482 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:10.482 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:10.482 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:10.482 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:46:10.482 7f3bf66c9588 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:10.773 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51493536, partition_space_available_1k=29596948, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=169272, cpu_share=0.79, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=857156, cpu_share=0.02, mem_res=767844, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42840, cpu_share=0.01, mem_res=39660, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15519596, free=9106460, buffers=998008, cached=7041056, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.76, five_min_avg=4.43, fifteen_min_avg=3.94, cpu_share=5.5, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:46:10.774 7f3bf899deb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:10.775 7f3bf899deb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51493536 partition_space_available_1k = 29596948 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 169272 cpu_share = 0.79 mem_res = 107744 >>kafka : << mem_virt = 857156 cpu_share = 0.02 mem_res = 767844 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42840 cpu_share = 0.01 mem_res = 39660 >> } system_mem_usage = << total = 32665120 used = 15519596 free = 9106460 buffers = 998008 cached = 7041056 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 5.76 five_min_avg = 4.43 fifteen_min_avg = 3.94 cpu_share = 5.5 node_type = analytics-alarm-node >> >> 06/02/2025 02:46:10.775 7f3bf899deb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:14.482 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:16.626 7f3bf66e84e0 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:46:19.488 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:19.488 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:19.488 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:19.488 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:19.488 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:19.488 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:19.489 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:19.489 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:19.489 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:19.489 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:19.489 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:19.489 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:46:19.489 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:23.491 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:28.497 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:28.497 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:28.497 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:28.498 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:28.498 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:28.498 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:28.498 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:28.499 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:28.499 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:28.499 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:28.499 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:28.499 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:46:28.499 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:32.499 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:37.505 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:37.505 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:37.505 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:37.505 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:37.505 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:37.506 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:37.506 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:37.506 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:37.506 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:37.506 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:37.506 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:37.506 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:46:37.506 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:41.509 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:46.512 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:46.512 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:46.512 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:46.513 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:46.513 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:46.513 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:46.513 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:46.513 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:46.514 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:46.514 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:46.514 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:46.514 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:46:46.514 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:50.516 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:55.522 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:55.523 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:55.523 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:55.523 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:55.523 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:55.523 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:55.524 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:55.524 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:55.524 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:55.524 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:55.524 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:55.524 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:46:55.524 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:59.526 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:04.533 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:04.533 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:04.533 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:04.533 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:04.534 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:04.534 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:04.534 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:04.534 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:04.534 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:04.535 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:04.535 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:04.535 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:47:04.535 7f3bfaa9fac8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:08.533 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:10.752 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51493008, partition_space_available_1k=29597476, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168220, cpu_share=0.9, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=857188, cpu_share=0.03, mem_res=767844, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42896, cpu_share=0.01, mem_res=39660, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15435252, free=9191832, buffers=998012, cached=7040024, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.89, five_min_avg=4.11, fifteen_min_avg=3.86, cpu_share=2.88, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:47:13.537 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:13.537 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:13.537 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:13.537 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:13.537 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:13.537 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:13.537 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51493008 partition_space_available_1k = 29597476 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 168220 cpu_share = 0.9 mem_res = 107744 >>kafka : << mem_virt = 857188 cpu_share = 0.03 mem_res = 767844 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42896 cpu_share = 0.01 mem_res = 39660 >> } system_mem_usage = << total = 32665120 used = 15435252 free = 9191832 buffers = 998012 cached = 7040024 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 3.89 five_min_avg = 4.11 fifteen_min_avg = 3.86 cpu_share = 2.88 node_type = analytics-alarm-node >> >> 06/02/2025 02:47:13.537 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:13.537 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:13.538 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:13.538 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:13.538 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:13.538 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:13.538 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:13.538 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:47:13.538 7f3bfae05d68 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:17.539 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:22.545 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:22.545 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:22.546 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:22.546 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:22.546 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:22.546 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:22.546 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:22.546 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:22.547 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:22.547 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:22.547 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:22.547 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:47:22.547 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:26.549 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:27.083 7f3bf66c9ac8 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:47:31.552 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:31.552 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:31.552 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:31.552 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:31.552 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:31.552 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:31.553 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:31.554 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:31.554 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:31.554 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:31.554 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:31.554 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:47:31.554 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:35.556 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:40.562 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:40.562 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:40.563 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:40.563 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:40.563 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:40.563 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:40.563 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:40.564 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:40.564 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:40.564 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:40.564 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:40.564 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:47:40.564 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:44.565 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:49.569 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:49.569 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:49.569 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:49.570 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:49.570 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:49.570 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:49.570 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:49.570 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:49.570 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:49.570 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:49.571 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:49.571 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:47:49.571 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:53.573 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:58.579 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:58.579 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:58.579 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:58.579 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:58.579 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:58.579 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:58.580 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:58.580 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:58.580 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:58.580 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:58.580 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:58.580 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:47:58.580 7f3bfadc4eb8 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:02.581 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:48:07.584 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:48:07.584 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:48:07.584 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:07.584 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:48:07.584 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:07.584 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:48:07.585 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:48:07.585 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:07.585 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:48:07.585 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:07.585 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:07.585 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:48:07.585 7f3bfffa1780 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:10.756 7f3bfadbf6c8 [contrail-analytics-alarm-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51494544, partition_space_available_1k=29595940, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-alarm-gen': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167280, cpu_share=0.89, mem_res=107744, _EventManager__key='contrail-alarm-gen'), 'kafka': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=857176, cpu_share=0.02, mem_res=767856, _EventManager__key='kafka'), 'contrail-analytics-alarm-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42888, cpu_share=0.01, mem_res=39660, _EventManager__key='contrail-analytics-alarm-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15413296, free=9210240, buffers=998016, cached=7043568, node_type='analytics-alarm-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.37, five_min_avg=3.95, fifteen_min_avg=3.82, cpu_share=2.88, node_type='analytics-alarm-node'), running_package_version=None, installed_package_version=None, _table='ObjectAnalyticsAlarmInfo') 06/02/2025 02:48:10.756 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:10.757 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51494544 partition_space_available_1k = 29595940 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 167280 cpu_share = 0.89 mem_res = 107744 >>kafka : << mem_virt = 857176 cpu_share = 0.02 mem_res = 767856 >>contrail-analytics-alarm-nodemgr : << mem_virt = 42888 cpu_share = 0.01 mem_res = 39660 >> } system_mem_usage = << total = 32665120 used = 15413296 free = 9210240 buffers = 998016 cached = 7043568 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 3.37 five_min_avg = 3.95 fifteen_min_avg = 3.82 cpu_share = 2.88 node_type = analytics-alarm-node >> >> 06/02/2025 02:48:10.757 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:11.585 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:48:16.587 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:48:16.588 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:48:16.588 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:16.588 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:48:16.588 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:16.588 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:48:16.588 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:48:16.589 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:16.589 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:48:16.589 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:16.589 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:16.589 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:48:16.589 7f3bf89c3a20 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:20.592 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:48:25.598 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:48:25.598 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:48:25.598 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:25.598 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:48:25.598 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:25.598 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:48:25.599 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:48:25.599 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:25.599 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:48:25.599 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:25.599 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:25.599 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1748829965371080 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 06/02/2025 02:48:25.599 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:28.910 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [DEBUG]: send sync_uve in the [ObjectAnalyticsAlarmInfo:NodeStatusUVE] map 06/02/2025 02:48:29.600 7f3bf89b1c18 [contrail-analytics-alarm-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]