10/21/2024 02:00:59.789 7fa9de6f1f20 [contrail-analytics-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 10/21/2024 02:00:59.789 7fa9de6f1f20 [contrail-analytics-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 10/21/2024 02:00:59.789 7fa9de6f1f20 [contrail-analytics-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 10/21/2024 02:00:59.798 7fa9d80bd248 [contrail-analytics-nodemgr] [ERROR]: Cannot write http_port 8104 to /tmp/contrail-analytics-nodemgr.0.http_port 10/21/2024 02:00:59.798 7fa9d80bd248 [contrail-analytics-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8104 10/21/2024 02:00:59.799 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvStart] in state[none] 10/21/2024 02:00:59.800 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: Add uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:00:59.800 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/21/2024 02:00:59.800 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 10/21/2024 02:00:59.801 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:59.801 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/21/2024 02:00:59.801 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:59.801 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:59.801 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 10/21/2024 02:00:59.801 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:59.929 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'contrail-collector', 'group': 'contrail-collector', 'pid': 17472055216760268969030948950915866064312433617419957233718026941399936603410, 'start': '1729476057000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-analytics-api', 'group': 'contrail-analytics-api', 'pid': 112775465174853599478677589011283202090266318502086182956880506469859232054199, 'start': '1729476057000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-analytics-nodemgr', 'group': 'contrail-analytics-nodemgr', 'pid': 113591545227550318078584978066081264283478845263433546432497774919136584726302, 'start': '1729476057000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 10/21/2024 02:01:00.120 7fa9de6f1f20 [contrail-analytics-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-analytics-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-439-1', _node_type='Analytics', _instance_id='0', _timestamp=1729476060119934, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-analytics-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo'), _seqnum=0) 10/21/2024 02:01:00.120 7fa9de6f1f20 [contrail-analytics-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-analytics-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-439-1', _node_type='Analytics', _instance_id='0', _timestamp=1729476060120636, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-collector', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476057000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-analytics-api', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476057000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-analytics-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476057000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo'), _seqnum=0) 10/21/2024 02:01:00.126 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:01:00.127 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 10/21/2024 02:01:00.127 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:01:00.127 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:01:00.128 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/21/2024 02:01:00.128 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:01:00.128 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:01:00.128 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 deleted = False process_info = [ << process_name = contrail-collector process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476057000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-analytics-api process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476057000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-analytics-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476057000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 10/21/2024 02:01:00.128 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:01:03.801 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:01:03.809 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Session Event: TCP Connected 10/21/2024 02:01:03.810 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:01:03.810 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:01:03.810 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:01:03.810 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:01:03.810 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 10/21/2024 02:01:03.810 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 10/21/2024 02:01:03.811 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 10/21/2024 02:01:03.811 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 10/21/2024 02:01:03.811 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 10/21/2024 02:01:03.815 7fa9d8441848 [contrail-analytics-nodemgr] [DEBUG]: Extracted sandesh length: 0000001011 10/21/2024 02:01:03.815 7fa9d8441848 [contrail-analytics-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 10/21/2024 02:01:03.816 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 10/21/2024 02:01:03.816 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 10/21/2024 02:01:03.816 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:01:03.816 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/21/2024 02:01:03.817 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 10/21/2024 02:01:03.817 7fa9d80bd548 [contrail-analytics-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 10/21/2024 02:01:03.818 7fa9d80bd548 [contrail-analytics-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 10/21/2024 02:01:05.156 7fa9de6f1f20 [contrail-analytics-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-analytics-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-439-1', _node_type='Analytics', _instance_id='0', _timestamp=1729476065156304, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-collector', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476057000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-analytics-api', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476057000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-analytics-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476057000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo'), _seqnum=0) 10/21/2024 02:01:05.156 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51075112, partition_space_available_1k=30015372, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=24492, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2856, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53412, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11493256, free=607472, buffers=762092, cached=19802296, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.33, five_min_avg=4.33, fifteen_min_avg=2.93, cpu_share=0, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:01:09.366 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:01:33.372 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:01:50.702 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:02:05.172 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51366420, partition_space_available_1k=29724064, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27192, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=74928, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45864, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14435228, free=12211980, buffers=591112, cached=5426796, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=14.41, five_min_avg=7.38, fifteen_min_avg=4.09, cpu_share=9.75, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:02:11.402 7fa9d80bd548 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:02:45.238 7fa9d80bd548 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:03:05.183 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51417892, partition_space_available_1k=29672592, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27228, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75176, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45888, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14952944, free=11374360, buffers=620392, cached=5717420, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=11.12, five_min_avg=7.77, fifteen_min_avg=4.44, cpu_share=6.62, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:03:16.443 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:03:30.607 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:04:05.191 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51430576, partition_space_available_1k=29659908, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27240, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75252, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45892, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14605856, free=11676508, buffers=622768, cached=5759984, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.54, five_min_avg=6.78, fifteen_min_avg=4.3, cpu_share=3.62, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:04:49.630 7fa9d80bd548 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:05:04.889 7fa9d80bd548 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:05:05.203 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51438096, partition_space_available_1k=29652388, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27260, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75340, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45920, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14690564, free=11512556, buffers=626044, cached=5835952, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.65, five_min_avg=6.54, fifteen_min_avg=4.37, cpu_share=2.5, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:05:18.982 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:05:58.910 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:06:05.233 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51444128, partition_space_available_1k=29646356, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27360, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75344, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45996, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14919668, free=10600072, buffers=673128, cached=6472248, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.13, five_min_avg=6.08, fifteen_min_avg=4.34, cpu_share=3.5, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:07:05.209 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51451032, partition_space_available_1k=29639452, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27388, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75376, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46004, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14652000, free=10233192, buffers=678364, cached=7101560, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.37, five_min_avg=6.57, fifteen_min_avg=4.64, cpu_share=4.62, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:08:05.213 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51454816, partition_space_available_1k=29635668, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27412, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75504, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46012, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14662164, free=10217376, buffers=678364, cached=7107212, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.8, five_min_avg=5.74, fifteen_min_avg=4.47, cpu_share=2.62, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:09:04.025 7fa9d80bd548 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:09:05.220 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51462572, partition_space_available_1k=29627912, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27424, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75508, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46004, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14712892, free=10157156, buffers=678460, cached=7116608, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.54, five_min_avg=5.02, fifteen_min_avg=4.3, cpu_share=2.62, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:09:17.941 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:09:34.941 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:09:59.742 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:10:05.222 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51466704, partition_space_available_1k=29623780, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27448, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75548, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46012, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14701432, free=10162556, buffers=678464, cached=7122664, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.98, five_min_avg=4.65, fifteen_min_avg=4.21, cpu_share=2.75, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:11:05.239 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51466424, partition_space_available_1k=29624060, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27480, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75612, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46016, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14639984, free=10222956, buffers=678476, cached=7123700, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.28, five_min_avg=4.17, fifteen_min_avg=4.08, cpu_share=2.75, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:12:05.230 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51476420, partition_space_available_1k=29614064, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27504, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75616, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46024, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14706416, free=10135336, buffers=678476, cached=7144888, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.47, five_min_avg=5.45, fifteen_min_avg=4.51, cpu_share=4.88, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:13:05.241 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51481564, partition_space_available_1k=29608920, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27528, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75668, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46032, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14699400, free=10135788, buffers=678484, cached=7151444, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.61, five_min_avg=4.88, fifteen_min_avg=4.37, cpu_share=2.75, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:13:14.925 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:13:28.741 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:13:45.888 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:14:05.240 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51485676, partition_space_available_1k=29604808, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27540, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75724, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46032, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14755320, free=10071520, buffers=678492, cached=7159784, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.25, five_min_avg=4.46, fifteen_min_avg=4.26, cpu_share=3.0, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:15:05.242 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51489620, partition_space_available_1k=29600864, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27560, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75756, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46036, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14786388, free=10034712, buffers=678528, cached=7165488, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.46, five_min_avg=4.0, fifteen_min_avg=4.11, cpu_share=2.62, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:15:31.896 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:16:05.254 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51493276, partition_space_available_1k=29597208, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27580, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75784, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46044, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14805272, free=10009852, buffers=678536, cached=7171456, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.02, five_min_avg=3.57, fifteen_min_avg=3.95, cpu_share=2.62, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:17:05.248 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51497236, partition_space_available_1k=29593248, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27608, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75844, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46040, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14790884, free=10005504, buffers=679496, cached=7189232, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.14, five_min_avg=3.34, fifteen_min_avg=3.85, cpu_share=2.75, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:17:25.233 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:17:40.486 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:17:57.770 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:18:05.256 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51511288, partition_space_available_1k=29579196, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27640, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75856, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46068, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14867276, free=9909204, buffers=679500, cached=7209136, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.08, five_min_avg=3.79, fifteen_min_avg=3.98, cpu_share=5.12, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:19:05.253 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51512480, partition_space_available_1k=29578004, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27648, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75904, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46068, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14843672, free=9922212, buffers=679616, cached=7219616, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.84, five_min_avg=3.5, fifteen_min_avg=3.87, cpu_share=3.12, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:20:05.253 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51515956, partition_space_available_1k=29574528, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27684, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75956, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46076, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14831800, free=9929504, buffers=679616, cached=7224196, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.42, five_min_avg=3.24, fifteen_min_avg=3.75, cpu_share=2.62, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:21:05.260 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51519524, partition_space_available_1k=29570960, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27696, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75952, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46072, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14814168, free=9942808, buffers=679620, cached=7228520, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.86, five_min_avg=3.14, fifteen_min_avg=3.68, cpu_share=2.5, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:21:20.120 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:22:05.251 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51523692, partition_space_available_1k=29566792, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27724, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75988, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46072, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14819488, free=9932404, buffers=679692, cached=7233532, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.75, five_min_avg=3.03, fifteen_min_avg=3.61, cpu_share=2.62, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:22:57.182 7fa9d80bd548 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:23:05.252 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51531512, partition_space_available_1k=29558972, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27748, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76120, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46092, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14905668, free=9836896, buffers=679820, cached=7242732, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.53, five_min_avg=2.89, fifteen_min_avg=3.52, cpu_share=2.62, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:23:10.898 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:23:34.745 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:24:05.267 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51538440, partition_space_available_1k=29552044, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27772, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76248, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46112, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14865020, free=9858548, buffers=679836, cached=7261712, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.62, five_min_avg=3.57, fifteen_min_avg=3.72, cpu_share=5.0, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:25:05.280 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51541956, partition_space_available_1k=29548528, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27792, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76280, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46116, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14877416, free=9835964, buffers=679844, cached=7271892, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.79, five_min_avg=3.2, fifteen_min_avg=3.58, cpu_share=2.62, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:26:05.306 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51545820, partition_space_available_1k=29544664, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27808, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76336, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46120, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14845576, free=9863148, buffers=679848, cached=7276544, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.86, five_min_avg=3.22, fifteen_min_avg=3.57, cpu_share=2.62, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:27:05.307 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51549980, partition_space_available_1k=29540504, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27820, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76356, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46116, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14862052, free=9841900, buffers=679852, cached=7281312, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.95, five_min_avg=2.88, fifteen_min_avg=3.43, cpu_share=2.62, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:27:12.867 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:27:50.591 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:28:04.413 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:28:05.338 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51553536, partition_space_available_1k=29536948, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27844, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76428, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46124, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=15296728, free=9395080, buffers=679852, cached=7293456, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.7, five_min_avg=3.48, fifteen_min_avg=3.58, cpu_share=3.38, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:28:23.121 7fa9d80bd548 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:29:05.322 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51560520, partition_space_available_1k=29529964, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27860, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76504, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46132, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14880276, free=9802204, buffers=679912, cached=7302724, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.8, five_min_avg=3.8, fifteen_min_avg=3.7, cpu_share=4.38, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:30:05.331 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51559344, partition_space_available_1k=29531140, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27876, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76504, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46132, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14908636, free=9773656, buffers=679912, cached=7302912, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.38, five_min_avg=3.55, fifteen_min_avg=3.61, cpu_share=2.5, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:31:05.344 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51563208, partition_space_available_1k=29527276, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27884, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76564, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46132, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14914580, free=9763320, buffers=679916, cached=7307300, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.77, five_min_avg=3.36, fifteen_min_avg=3.54, cpu_share=2.75, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:31:50.065 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:32:05.342 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51567520, partition_space_available_1k=29522964, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27904, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76600, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46132, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14954460, free=9723280, buffers=679916, cached=7307460, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.79, five_min_avg=3.3, fifteen_min_avg=3.51, cpu_share=2.75, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:32:31.698 7fa9d80bd548 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:32:45.673 7fa9d80bd548 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:33:02.661 7fa9d80bd548 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:33:05.348 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51673272, partition_space_available_1k=29417212, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27924, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76800, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46140, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14991052, free=9677532, buffers=679920, cached=7316612, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.28, five_min_avg=3.07, fifteen_min_avg=3.42, cpu_share=2.88, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:34:05.351 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51577684, partition_space_available_1k=29512800, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27944, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76860, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46140, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14957712, free=9785104, buffers=679920, cached=7242380, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.04, five_min_avg=3.75, fifteen_min_avg=3.63, cpu_share=5.0, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:35:05.360 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51576476, partition_space_available_1k=29514008, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27964, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76888, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46136, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14985320, free=9757096, buffers=679920, cached=7242780, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.57, five_min_avg=3.57, fifteen_min_avg=3.58, cpu_share=2.75, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:36:03.949 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:36:05.369 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51584856, partition_space_available_1k=29505628, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27984, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76884, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46140, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=15015560, free=9718184, buffers=679920, cached=7251452, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.5, five_min_avg=3.23, fifteen_min_avg=3.46, cpu_share=2.75, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:37:05.368 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51580224, partition_space_available_1k=29510260, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=27988, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76948, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46140, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14951712, free=9785584, buffers=679924, cached=7247896, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.16, five_min_avg=2.99, fifteen_min_avg=3.36, cpu_share=2.75, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:38:05.177 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:38:05.374 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51588504, partition_space_available_1k=29501980, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=28000, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76980, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46144, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=15008884, free=9718268, buffers=679924, cached=7258040, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.15, five_min_avg=2.83, fifteen_min_avg=3.27, cpu_share=2.88, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:38:19.193 7fa9d80bd548 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:38:36.537 7fa9d80bd548 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:39:05.376 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51583732, partition_space_available_1k=29506752, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=28012, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76976, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46148, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14969656, free=9760008, buffers=679924, cached=7255528, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.65, five_min_avg=2.87, fifteen_min_avg=3.26, cpu_share=3.25, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:39:35.901 7fa9d8441848 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connection Closed 10/21/2024 02:39:35.901 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 10/21/2024 02:39:40.906 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:40.907 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 10/21/2024 02:39:40.907 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:40.907 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 10/21/2024 02:39:40.907 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:40.907 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:40.908 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:40.908 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:40.908 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:40.908 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:40.908 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:40.908 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:39:40.908 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:44.908 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:49.914 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:49.914 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:49.914 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:49.914 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:49.914 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:49.914 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:49.915 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:49.915 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:49.915 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:49.915 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:49.915 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:49.915 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:39:49.915 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:53.915 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:58.933 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:58.934 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:58.934 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:58.934 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:58.934 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:58.934 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:58.935 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:58.935 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:58.935 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:58.935 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:58.935 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:58.935 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:39:58.935 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:02.934 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:05.374 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51594380, partition_space_available_1k=29496104, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=28008, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=77032, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46160, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=15090564, free=9599416, buffers=681864, cached=7293272, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.79, five_min_avg=3.32, fifteen_min_avg=3.4, cpu_share=5.12, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:40:07.938 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:07.938 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:07.938 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:07.938 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:07.938 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:07.938 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:07.938 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51594380 partition_space_available_1k = 29496104 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-collector : << mem_virt = 28008 cpu_share = 0 mem_res = 0 >>contrail-analytics-api : << mem_virt = 77032 cpu_share = 0 mem_res = 0 >>contrail-analytics-nodemgr : << mem_virt = 46160 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 15090564 free = 9599416 buffers = 681864 cached = 7293272 node_type = analytics-node >> system_cpu_usage = << one_min_avg = 3.79 five_min_avg = 3.32 fifteen_min_avg = 3.4 cpu_share = 5.12 node_type = analytics-node >> >> 10/21/2024 02:40:07.938 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:07.939 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:07.939 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:07.939 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:07.939 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:07.939 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:07.939 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:07.939 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:40:07.940 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:11.941 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:16.945 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:16.945 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:16.945 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:16.945 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:16.945 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:16.945 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:16.946 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:16.946 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:16.946 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:16.946 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:16.946 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:16.946 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:40:16.946 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:20.949 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:25.953 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:25.953 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:25.953 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:25.953 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:25.953 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:25.953 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:25.954 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:25.954 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:25.954 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:25.954 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:25.954 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:25.954 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:40:25.954 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:29.955 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:30.749 7fa9d8441848 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:40:34.960 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:34.960 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:34.960 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:34.960 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:34.961 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:34.961 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:34.961 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:34.961 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:34.962 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:34.962 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:34.962 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:34.962 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:40:34.962 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:38.963 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:43.965 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:43.965 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:43.965 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:43.966 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:43.966 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:43.966 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:43.966 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:43.966 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:43.966 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:43.967 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:43.967 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:43.967 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:40:43.967 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:47.966 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:52.971 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:52.971 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:52.972 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:52.972 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:52.972 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:52.972 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:52.972 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:52.972 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:52.973 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:52.973 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:52.973 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:52.973 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:40:52.973 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:56.974 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:01.976 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:01.976 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:01.976 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:01.976 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:01.976 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:01.977 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:01.977 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:01.977 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:01.977 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:01.978 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:01.978 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:01.978 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:41:01.978 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:05.384 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51599384, partition_space_available_1k=29491100, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=28008, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=77064, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46176, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=15008756, free=9673480, buffers=681916, cached=7300964, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.41, five_min_avg=3.02, fifteen_min_avg=3.29, cpu_share=2.88, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:41:05.385 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:05.385 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51599384 partition_space_available_1k = 29491100 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-collector : << mem_virt = 28008 cpu_share = 0 mem_res = 0 >>contrail-analytics-api : << mem_virt = 77064 cpu_share = 0 mem_res = 0 >>contrail-analytics-nodemgr : << mem_virt = 46176 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 15008756 free = 9673480 buffers = 681916 cached = 7300964 node_type = analytics-node >> system_cpu_usage = << one_min_avg = 2.41 five_min_avg = 3.02 fifteen_min_avg = 3.29 cpu_share = 2.88 node_type = analytics-node >> >> 10/21/2024 02:41:05.385 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:05.976 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:10.981 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:10.981 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:10.981 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:10.982 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:10.982 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:10.982 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:10.982 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:10.982 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:10.983 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:10.983 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:10.983 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:10.983 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:41:10.983 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:14.982 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:19.985 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:19.985 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:19.985 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:19.985 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:19.985 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:19.985 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:19.986 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:19.986 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:19.986 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:19.986 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:19.986 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:19.986 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:41:19.986 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:23.987 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:28.989 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:28.989 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:28.989 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:28.990 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:28.990 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:28.990 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:28.990 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:28.991 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:28.991 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:28.991 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:28.991 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:28.991 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:41:28.991 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:32.993 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:37.998 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:37.998 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:37.998 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:37.998 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:37.998 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:37.998 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:37.999 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:37.999 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:37.999 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:37.999 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:37.999 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:38.000 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:41:38.000 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:42.001 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:47.005 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:47.005 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:47.005 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:47.005 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:47.005 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:47.006 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:47.006 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:47.006 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:47.006 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:47.006 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:47.007 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:47.007 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:41:47.007 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:51.007 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:56.013 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:56.013 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:56.013 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:56.013 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:56.013 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:56.013 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:56.014 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:56.014 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:56.014 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:56.014 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:56.014 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:56.014 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:41:56.014 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:00.013 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:05.016 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:05.016 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:05.017 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:05.017 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:05.017 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:05.017 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:05.018 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:05.018 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:05.018 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:05.018 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:05.018 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:05.018 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:42:05.018 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:05.375 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51598120, partition_space_available_1k=29492364, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=28008, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=77076, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46212, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14986156, free=9696868, buffers=681916, cached=7300176, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.31, five_min_avg=2.88, fifteen_min_avg=3.22, cpu_share=2.88, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:42:05.376 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:05.376 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51598120 partition_space_available_1k = 29492364 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-collector : << mem_virt = 28008 cpu_share = 0 mem_res = 0 >>contrail-analytics-api : << mem_virt = 77076 cpu_share = 0 mem_res = 0 >>contrail-analytics-nodemgr : << mem_virt = 46212 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14986156 free = 9696868 buffers = 681916 cached = 7300176 node_type = analytics-node >> system_cpu_usage = << one_min_avg = 2.31 five_min_avg = 2.88 fifteen_min_avg = 3.22 cpu_share = 2.88 node_type = analytics-node >> >> 10/21/2024 02:42:05.376 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:09.017 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:14.019 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:14.019 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:14.019 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:14.019 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:14.019 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:14.019 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:14.020 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:14.020 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:14.020 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:14.020 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:14.020 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:14.020 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:42:14.020 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:18.021 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:23.024 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:23.025 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:23.025 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:23.025 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:23.025 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:23.025 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:23.025 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:23.026 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:23.026 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:23.026 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:23.026 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:23.026 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:42:23.026 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:27.027 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:32.031 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:32.031 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:32.031 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:32.032 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:32.032 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:32.032 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:32.032 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:32.032 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:32.032 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:32.033 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:32.033 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:32.033 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:42:32.033 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:36.034 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:41.039 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:41.040 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:41.040 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:41.040 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:41.040 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:41.040 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:41.041 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:41.041 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:41.041 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:41.041 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:41.041 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:41.041 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:42:41.041 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:45.040 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:50.045 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:50.045 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:50.045 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:50.045 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:50.045 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:50.045 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:50.046 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:50.046 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:50.046 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:50.046 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:50.046 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:50.046 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:42:50.046 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:54.048 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:59.054 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:59.054 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:59.054 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:59.054 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:59.054 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:59.054 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:59.055 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:59.055 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:59.055 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:59.055 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:59.055 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:59.055 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:42:59.055 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:03.055 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:05.376 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51602108, partition_space_available_1k=29488376, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=28012, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=77132, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46256, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14983352, free=9694808, buffers=681916, cached=7305040, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.38, five_min_avg=2.77, fifteen_min_avg=3.16, cpu_share=2.75, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:43:08.060 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:08.060 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:08.060 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:08.060 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:08.061 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:08.061 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:08.061 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51602108 partition_space_available_1k = 29488376 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-collector : << mem_virt = 28012 cpu_share = 0 mem_res = 0 >>contrail-analytics-api : << mem_virt = 77132 cpu_share = 0 mem_res = 0 >>contrail-analytics-nodemgr : << mem_virt = 46256 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14983352 free = 9694808 buffers = 681916 cached = 7305040 node_type = analytics-node >> system_cpu_usage = << one_min_avg = 2.38 five_min_avg = 2.77 fifteen_min_avg = 3.16 cpu_share = 2.75 node_type = analytics-node >> >> 10/21/2024 02:43:08.061 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:08.061 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:08.062 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:08.062 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:08.062 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:08.062 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:08.063 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:08.063 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:43:08.063 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:12.063 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:17.066 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:17.066 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:17.066 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:17.066 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:17.066 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:17.066 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:17.067 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:17.067 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:17.067 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:17.067 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:17.067 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:17.067 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:43:17.068 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:21.069 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:26.075 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:26.075 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:26.075 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:26.075 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:26.075 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:26.075 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:26.076 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:26.076 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:26.076 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:26.076 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:26.076 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:26.076 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:43:26.076 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:30.075 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:35.081 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:35.081 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:35.081 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:35.081 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:35.081 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:35.082 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:35.082 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:35.082 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:35.083 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:35.083 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:35.083 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:35.083 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:43:35.083 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:39.082 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:44.088 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:44.088 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:44.088 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:44.088 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:44.088 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:44.089 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:44.089 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:44.089 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:44.089 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:44.089 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:44.089 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:44.090 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:43:44.090 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:48.090 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:53.095 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:53.095 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:53.095 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:53.096 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:53.096 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:53.096 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:53.097 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:53.097 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:53.097 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:53.097 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:53.097 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:53.097 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:43:53.097 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:57.099 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:44:01.623 7fa9d8441848 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:44:02.101 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:44:02.101 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:44:02.101 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:44:02.101 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:44:02.102 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:44:02.102 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:44:02.102 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:44:02.102 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:44:02.103 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-analytics-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:44:02.103 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:44:02.103 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:44:02.103 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Analytics:contrail-analytics-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8104 start_time = 1729476059800594 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:44:02.103 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:44:02.606 7fa9d80bd448 [contrail-analytics-nodemgr] [DEBUG]: send sync_uve in the [ObjectCollectorInfo:NodeStatusUVE] map 10/21/2024 02:44:05.379 7fa9de6f1f20 [contrail-analytics-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51709496, partition_space_available_1k=29380988, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-collector': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=28012, cpu_share=0, mem_res=0, _EventManager__key='contrail-collector'), 'contrail-analytics-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=77160, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-api'), 'contrail-analytics-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46268, cpu_share=0, mem_res=0, _EventManager__key='contrail-analytics-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=15117072, free=9396148, buffers=683932, cached=7467964, node_type='analytics-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.49, five_min_avg=2.7, fifteen_min_avg=3.11, cpu_share=3.0, node_type='analytics-node'), running_package_version=None, installed_package_version=None, _table='ObjectCollectorInfo') 10/21/2024 02:44:05.380 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:44:05.380 7fa9d80bd448 [contrail-analytics-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51709496 partition_space_available_1k = 29380988 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-collector : << mem_virt = 28012 cpu_share = 0 mem_res = 0 >>contrail-analytics-api : << mem_virt = 77160 cpu_share = 0 mem_res = 0 >>contrail-analytics-nodemgr : << mem_virt = 46268 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 15117072 free = 9396148 buffers = 683932 cached = 7467964 node_type = analytics-node >> system_cpu_usage = << one_min_avg = 2.49 five_min_avg = 2.7 fifteen_min_avg = 3.11 cpu_share = 3.0 node_type = analytics-node >> >> 10/21/2024 02:44:05.380 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:44:06.101 7fa9d80bd448 [contrail-analytics-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]