06/02/2025 02:06:40.261 7f251065d1c8 [contrail-config-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 06/02/2025 02:06:40.261 7f251065d1c8 [contrail-config-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 06/02/2025 02:06:40.262 7f251065d1c8 [contrail-config-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 06/02/2025 02:06:40.281 7f250cb22048 [contrail-config-nodemgr] [ERROR]: Cannot write http_port 8100 to /tmp/contrail-config-nodemgr.0.http_port 06/02/2025 02:06:40.284 7f250cb22048 [contrail-config-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8100 06/02/2025 02:06:40.285 7f250cb26978 [contrail-config-nodemgr] [INFO]: Processing event[EvStart] in state[none] 06/02/2025 02:06:40.287 7f250cb26978 [contrail-config-nodemgr] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:06:40.294 7f250cb26978 [contrail-config-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/02/2025 02:06:40.303 7f250cb26978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 06/02/2025 02:06:40.303 7f250cb26978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:40.303 7f250cb26978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/02/2025 02:06:40.303 7f250cb26978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:40.303 7f250cb26978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:40.303 7f250cb26978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 06/02/2025 02:06:40.303 7f250cb26978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:40.744 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'contrail-api', 'group': 'contrail-api', 'pid': 25869882663730920009205508036035664468233483330760372983739426970924507822097, 'start': '1748829996000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-schema', 'group': 'contrail-schema', 'pid': 11223618669602083936069821672345944010150992739191223877163873510497432317157, 'start': '1748829996000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-svc-monitor', 'group': 'contrail-svc-monitor', 'pid': 37858404784602226826818091014909179589515112064367285450212349376832405633253, 'start': '1748829996000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-device-manager', 'group': 'contrail-device-manager', 'pid': 13959229641042437166154983713679057693101352152056683541629816701907524491047, 'start': '1748829996000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-nodemgr', 'group': 'contrail-config-nodemgr', 'pid': 28703891004389132747009234106072278636815629862238214567026617024377474481374, 'start': '1748829996000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 06/02/2025 02:06:41.392 7f251065d1c8 [contrail-config-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-542-1', _node_type='Config', _instance_id='0', _timestamp=1748830001391999, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 06/02/2025 02:06:41.393 7f251065d1c8 [contrail-config-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-542-1', _node_type='Config', _instance_id='0', _timestamp=1748830001393247, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-api', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829996000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-schema', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829996000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-svc-monitor', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829996000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-device-manager', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829996000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829996000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-02 00:52:12.596288", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9311-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 06/02/2025 02:06:41.400 7f250cb38eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:41.401 7f250cb38eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-02 00:52:12.596288", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9311-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 06/02/2025 02:06:41.402 7f250cb38eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:41.402 7f250cb38eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:41.402 7f250cb38eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/02/2025 02:06:41.402 7f250cb38eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:41.402 7f250cb38eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:41.402 7f250cb38eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 deleted = False process_info = [ << process_name = contrail-api process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829996000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-schema process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829996000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-svc-monitor process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829996000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-device-manager process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829996000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829996000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-02 00:52:12.596288", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9311-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 06/02/2025 02:06:41.410 7f250cb38eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:44.288 7f250c9c6d68 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:06:47.115 7f250dd730f0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:06:49.293 7f250c9c6d68 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:06:49.294 7f250c9c6d68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:06:49.294 7f250c9c6d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:49.294 7f250c9c6d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:06:49.294 7f250c9c6d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:49.294 7f250c9c6d68 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:06:49.294 7f250c9c6d68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:06:49.299 7f250c9c6d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:49.303 7f250c9c6d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:06:49.303 7f250c9c6d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:49.303 7f250c9c6d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:49.303 7f250c9c6d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:06:49.303 7f250c9c6d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:50.488 7f251065d1c8 [contrail-config-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-542-1', _node_type='Config', _instance_id='0', _timestamp=1748830010488754, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-api', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829996000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-schema', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829996000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-svc-monitor', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829996000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-device-manager', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829996000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748829996000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-02 00:52:12.596288", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9311-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 06/02/2025 02:06:50.489 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51177208, partition_space_available_1k=29913276, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1468, cpu_share=0.0, mem_res=800, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1460, cpu_share=0.0, mem_res=804, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1456, cpu_share=0.0, mem_res=800, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1592, cpu_share=0.0, mem_res=796, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=41780, cpu_share=0.0, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=13789644, free=12993732, buffers=933372, cached=4948372, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=7.92, five_min_avg=4.26, fifteen_min_avg=2.79, cpu_share=0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:06:50.489 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:50.489 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 deleted = False process_info = [ << process_name = contrail-api process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829996000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-schema process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829996000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-svc-monitor process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829996000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-device-manager process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829996000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1748829996000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-02 00:52:12.596288", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9311-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 06/02/2025 02:06:50.489 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:50.489 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:50.490 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51177208 partition_space_available_1k = 29913276 percentage_partition_space_used = 63 >> } all_core_file_list = [ ] process_mem_cpu_usage = { contrail-api : << mem_virt = 1468 cpu_share = 0.0 mem_res = 800 >>contrail-schema : << mem_virt = 1460 cpu_share = 0.0 mem_res = 804 >>contrail-svc-monitor : << mem_virt = 1456 cpu_share = 0.0 mem_res = 800 >>contrail-device-manager : << mem_virt = 1592 cpu_share = 0.0 mem_res = 796 >>contrail-config-nodemgr : << mem_virt = 41780 cpu_share = 0.0 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 13789644 free = 12993732 buffers = 933372 cached = 4948372 node_type = config-node >> system_cpu_usage = << one_min_avg = 7.92 five_min_avg = 4.26 fifteen_min_avg = 2.79 cpu_share = 0 node_type = config-node >> >> 06/02/2025 02:06:50.490 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:53.296 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:06:58.301 7f251047ab70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:06:58.302 7f251047ab70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:06:58.302 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:58.302 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:06:58.302 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:06:58.302 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:06:58.303 7f251047ab70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:06:58.303 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:58.303 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:06:58.304 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:58.304 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:58.304 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:06:58.304 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:06:59.512 7f251047ab70 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:07:02.305 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:07.313 7f251047ab70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:07.313 7f251047ab70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:07.313 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:07.314 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:07.314 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:07.314 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:07.315 7f251047ab70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:07.315 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:07.315 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:07.315 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:07.316 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:07.316 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:07:07.316 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:11.317 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:12.730 7f250dd6c4e0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:07:16.321 7f251047ab70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:16.321 7f251047ab70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:16.321 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:16.322 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:16.322 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:16.322 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:16.322 7f251047ab70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:16.322 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:16.322 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:16.322 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:16.322 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:16.323 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:07:16.323 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:20.324 7f250cb26b70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:25.331 7f250cb26b70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:25.331 7f250cb26b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:25.331 7f250cb26b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:25.332 7f250cb26b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:25.332 7f250cb26b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:25.332 7f250cb26b70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:25.332 7f250cb26b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:25.338 7f250cb26b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:25.338 7f250cb26b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:25.338 7f250cb26b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:25.338 7f250cb26b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:25.338 7f250cb26b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:07:25.338 7f250cb26b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:29.333 7f250dd6c978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:34.353 7f250dd6c978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:34.353 7f250dd6c978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:34.353 7f250dd6c978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:34.353 7f250dd6c978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:34.353 7f250dd6c978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:34.354 7f250dd6c978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:34.354 7f250dd6c978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:34.354 7f250dd6c978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:34.354 7f250dd6c978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:34.354 7f250dd6c978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:34.354 7f250dd6c978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:34.355 7f250dd6c978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:07:34.355 7f250dd6c978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:35.252 7f250dd6c978 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:07:38.354 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:43.357 7f251687fe10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:43.357 7f251687fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:43.357 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:43.358 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:43.358 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:43.358 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:43.358 7f251687fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:43.358 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:43.358 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:43.359 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:43.359 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:43.359 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:07:43.359 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:47.358 7f2510475630 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:07:50.474 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51259244, partition_space_available_1k=29831240, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102240, cpu_share=0.11, mem_res=92304, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=87984, cpu_share=0.06, mem_res=78072, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=96600, cpu_share=0.08, mem_res=85108, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=201692, cpu_share=0.11, mem_res=175096, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=41856, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=14979532, free=10566660, buffers=984084, cached=6134844, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=22.3, five_min_avg=8.88, fifteen_min_avg=4.46, cpu_share=10.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:07:52.362 7f2510475630 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:07:52.362 7f2510475630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:07:52.362 7f2510475630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:52.363 7f2510475630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:07:52.363 7f2510475630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:52.363 7f2510475630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:52.363 7f2510475630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51259244 partition_space_available_1k = 29831240 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102240 cpu_share = 0.11 mem_res = 92304 >>contrail-schema : << mem_virt = 87984 cpu_share = 0.06 mem_res = 78072 >>contrail-svc-monitor : << mem_virt = 96600 cpu_share = 0.08 mem_res = 85108 >>contrail-device-manager : << mem_virt = 201692 cpu_share = 0.11 mem_res = 175096 >>contrail-config-nodemgr : << mem_virt = 41856 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 14979532 free = 10566660 buffers = 984084 cached = 6134844 node_type = config-node >> system_cpu_usage = << one_min_avg = 22.3 five_min_avg = 8.88 fifteen_min_avg = 4.46 cpu_share = 10.5 node_type = config-node >> >> 06/02/2025 02:07:52.363 7f2510475630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:07:52.363 7f2510475630 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:07:52.363 7f2510475630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:07:52.363 7f2510475630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:52.364 7f2510475630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:07:52.364 7f2510475630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:52.364 7f2510475630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:52.364 7f2510475630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:07:52.364 7f2510475630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:07:56.366 7f251047cb70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:01.370 7f251047cb70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:01.370 7f251047cb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:01.370 7f251047cb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:01.370 7f251047cb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:01.371 7f251047cb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:01.371 7f251047cb70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:01.371 7f251047cb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:01.371 7f251047cb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:01.371 7f251047cb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:01.371 7f251047cb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:01.371 7f251047cb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:01.371 7f251047cb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:08:01.372 7f251047cb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:05.374 7f250d592630 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:09.714 7f2511b3d0f0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:08:10.376 7f250d592630 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:10.376 7f250d592630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:10.376 7f250d592630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:10.377 7f250d592630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:10.377 7f250d592630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:10.377 7f250d592630 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:10.378 7f250d592630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:10.379 7f250d592630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:10.379 7f250d592630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:10.379 7f250d592630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:10.379 7f250d592630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:10.379 7f250d592630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:08:10.379 7f250d592630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:14.378 7f251047e588 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:19.385 7f251047e588 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:19.386 7f251047e588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:19.386 7f251047e588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:19.386 7f251047e588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:19.386 7f251047e588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:19.386 7f251047e588 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:19.386 7f251047e588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:19.387 7f251047e588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:19.387 7f251047e588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:19.387 7f251047e588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:19.387 7f251047e588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:19.387 7f251047e588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:08:19.387 7f251047e588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:23.389 7f251047e588 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:24.909 7f2510473f60 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:08:28.393 7f251047e588 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:28.394 7f251047e588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:28.394 7f251047e588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:28.394 7f251047e588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:28.394 7f251047e588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:28.394 7f251047e588 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:28.395 7f251047e588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:28.395 7f251047e588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:28.395 7f251047e588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:28.395 7f251047e588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:28.395 7f251047e588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:28.396 7f251047e588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:08:28.396 7f251047e588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:32.396 7f250cb38978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:37.402 7f250cb38978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:37.402 7f250cb38978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:37.403 7f250cb38978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:37.403 7f250cb38978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:37.403 7f250cb38978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:37.403 7f250cb38978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:37.403 7f250cb38978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:37.403 7f250cb38978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:37.404 7f250cb38978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:37.404 7f250cb38978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:37.404 7f250cb38978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:37.404 7f250cb38978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:08:37.404 7f250cb38978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:41.407 7f250dd8f198 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:46.412 7f250dd8f198 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:46.412 7f250dd8f198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:46.412 7f250dd8f198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:46.412 7f250dd8f198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:46.412 7f250dd8f198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:46.413 7f250dd8f198 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:46.413 7f250dd8f198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:46.414 7f250dd8f198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:46.414 7f250dd8f198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:46.414 7f250dd8f198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:46.414 7f250dd8f198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:46.414 7f250dd8f198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:08:46.414 7f250dd8f198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:46.711 7f251047ccc0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:08:50.415 7f2510472ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:08:50.479 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51264224, partition_space_available_1k=29826260, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101100, cpu_share=0.16, mem_res=96172, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84404, cpu_share=0.02, mem_res=78644, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93240, cpu_share=0.01, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=187880, cpu_share=0.07, mem_res=175248, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=41884, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15219300, free=11267492, buffers=990312, cached=5188016, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=13.5, five_min_avg=8.83, fifteen_min_avg=4.72, cpu_share=6.0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:08:55.421 7f2510472ac8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:08:55.421 7f2510472ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:08:55.421 7f2510472ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:55.421 7f2510472ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:08:55.422 7f2510472ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:55.422 7f2510472ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:55.422 7f2510472ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51264224 partition_space_available_1k = 29826260 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 101100 cpu_share = 0.16 mem_res = 96172 >>contrail-schema : << mem_virt = 84404 cpu_share = 0.02 mem_res = 78644 >>contrail-svc-monitor : << mem_virt = 93240 cpu_share = 0.01 mem_res = 85100 >>contrail-device-manager : << mem_virt = 187880 cpu_share = 0.07 mem_res = 175248 >>contrail-config-nodemgr : << mem_virt = 41884 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15219300 free = 11267492 buffers = 990312 cached = 5188016 node_type = config-node >> system_cpu_usage = << one_min_avg = 13.5 five_min_avg = 8.83 fifteen_min_avg = 4.72 cpu_share = 6.0 node_type = config-node >> >> 06/02/2025 02:08:55.422 7f2510472ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:08:55.422 7f2510472ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:08:55.423 7f2510472ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:08:55.423 7f2510472ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:55.423 7f2510472ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:08:55.423 7f2510472ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:55.424 7f2510472ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:55.424 7f2510472ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:08:55.424 7f2510472ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:08:57.155 7f250d57d240 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:08:59.422 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:04.427 7f2511b3d0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:04.427 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:04.427 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:04.428 7f2511b3d0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:04.428 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:04.428 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:04.428 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:04.428 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:04.428 7f2511b3d0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:04.428 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:04.428 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:04.429 7f2511b3d0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:09:04.429 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:08.431 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:13.437 7f2511b3d0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:13.438 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:13.438 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:13.438 7f2511b3d0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:13.438 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:13.438 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:13.438 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:13.438 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:13.438 7f2511b3d0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:13.438 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:13.439 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:13.439 7f2511b3d0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:09:13.439 7f2511b3d0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:17.440 7f250d592048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:22.445 7f250d592048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:22.445 7f250d592048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:22.445 7f250d592048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:22.445 7f250d592048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:22.445 7f250d592048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:22.445 7f250d592048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:22.446 7f250d592048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:22.446 7f250d592048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:22.446 7f250d592048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:22.446 7f250d592048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:22.446 7f250d592048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:22.446 7f250d592048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:09:22.446 7f250d592048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:26.449 7f250d592048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:31.453 7f250d592048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:31.453 7f250d592048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:31.453 7f250d592048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:31.453 7f250d592048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:31.453 7f250d592048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:31.453 7f250d592048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:31.453 7f250d592048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:31.454 7f250d592048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:31.454 7f250d592048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:31.454 7f250d592048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:31.454 7f250d592048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:31.454 7f250d592048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:09:31.454 7f250d592048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:35.457 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:40.463 7f2510660240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:40.463 7f2510660240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:40.464 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:40.464 7f2510660240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:40.464 7f2510660240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:40.464 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:40.464 7f2510660240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:40.464 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:40.464 7f2510660240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:40.465 7f2510660240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:40.465 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:40.465 7f2510660240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:09:40.465 7f2510660240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:44.464 7f250dd8bb70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:49.467 7f250dd8bb70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:49.468 7f250dd8bb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:49.468 7f250dd8bb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:49.468 7f250dd8bb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:49.468 7f250dd8bb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:49.468 7f250dd8bb70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:49.469 7f250dd8bb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:49.469 7f250dd8bb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:49.469 7f250dd8bb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:49.469 7f250dd8bb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:49.469 7f250dd8bb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:49.469 7f250dd8bb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:09:49.469 7f250dd8bb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:50.476 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51298904, partition_space_available_1k=29791580, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101636, cpu_share=0.03, mem_res=96536, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84492, cpu_share=0.0, mem_res=78712, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93492, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188780, cpu_share=0.02, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=41880, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15104872, free=11291572, buffers=990872, cached=5277804, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=7.8, five_min_avg=8.1, fifteen_min_avg=4.73, cpu_share=4.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:09:50.476 7f2510460eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:50.477 7f2510460eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51298904 partition_space_available_1k = 29791580 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 101636 cpu_share = 0.03 mem_res = 96536 >>contrail-schema : << mem_virt = 84492 cpu_share = 0.0 mem_res = 78712 >>contrail-svc-monitor : << mem_virt = 93492 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 188780 cpu_share = 0.02 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 41880 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15104872 free = 11291572 buffers = 990872 cached = 5277804 node_type = config-node >> system_cpu_usage = << one_min_avg = 7.8 five_min_avg = 8.1 fifteen_min_avg = 4.73 cpu_share = 4.38 node_type = config-node >> >> 06/02/2025 02:09:50.477 7f2510460eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:53.469 7f2510460eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:09:58.475 7f2510460eb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:09:58.475 7f2510460eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:09:58.475 7f2510460eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:58.475 7f2510460eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:09:58.475 7f2510460eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:09:58.475 7f2510460eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:09:58.476 7f2510460eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:09:58.476 7f2510460eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:58.476 7f2510460eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:09:58.476 7f2510460eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:58.477 7f2510460eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:09:58.477 7f2510460eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:09:58.477 7f2510460eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:02.478 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:07.481 7f250e226828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:07.481 7f250e226828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:07.481 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:07.481 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:07.481 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:07.481 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:07.482 7f250e226828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:07.482 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:07.482 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:07.482 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:07.482 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:07.482 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:10:07.482 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:11.485 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:16.489 7f250e226828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:16.489 7f250e226828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:16.489 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:16.489 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:16.489 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:16.489 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:16.490 7f250e226828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:16.490 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:16.490 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:16.490 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:16.490 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:16.490 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:10:16.490 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:18.130 7f250e226828 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:10:20.491 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:25.497 7f250e226828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:25.497 7f250e226828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:25.497 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:25.497 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:25.497 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:25.498 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:25.498 7f250e226828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:25.498 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:25.498 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:25.498 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:25.498 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:25.499 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:10:25.499 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:28.826 7f250e226828 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:10:29.497 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:34.505 7f250e226828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:34.505 7f250e226828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:34.505 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:34.505 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:34.505 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:34.505 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:34.506 7f250e226828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:34.506 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:34.506 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:34.506 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:34.506 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:34.506 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:10:34.506 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:38.508 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:43.511 7f250e226828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:43.511 7f250e226828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:43.511 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:43.511 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:43.511 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:43.511 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:43.512 7f250e226828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:43.512 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:43.512 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:43.512 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:43.512 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:43.512 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:10:43.512 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:47.512 7f250dd6d198 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:50.471 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51292940, partition_space_available_1k=29797544, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101708, cpu_share=0.0, mem_res=96536, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84584, cpu_share=0.0, mem_res=78748, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93548, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188784, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=41936, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15139108, free=11118356, buffers=991144, cached=5416512, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.11, five_min_avg=6.96, fifteen_min_avg=4.55, cpu_share=3.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:10:52.517 7f250dd6d198 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:10:52.517 7f250dd6d198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:10:52.517 7f250dd6d198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:52.517 7f250dd6d198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:10:52.517 7f250dd6d198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:52.517 7f250dd6d198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:52.517 7f250dd6d198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51292940 partition_space_available_1k = 29797544 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 101708 cpu_share = 0.0 mem_res = 96536 >>contrail-schema : << mem_virt = 84584 cpu_share = 0.0 mem_res = 78748 >>contrail-svc-monitor : << mem_virt = 93548 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 188784 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 41936 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15139108 free = 11118356 buffers = 991144 cached = 5416512 node_type = config-node >> system_cpu_usage = << one_min_avg = 4.11 five_min_avg = 6.96 fifteen_min_avg = 4.55 cpu_share = 3.12 node_type = config-node >> >> 06/02/2025 02:10:52.517 7f250dd6d198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:10:52.517 7f250dd6d198 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:10:52.518 7f250dd6d198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:10:52.518 7f250dd6d198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:52.518 7f250dd6d198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:10:52.518 7f250dd6d198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:52.518 7f250dd6d198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:52.518 7f250dd6d198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:10:52.518 7f250dd6d198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:10:56.521 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:10:59.902 7f250dd6d438 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:11:01.524 7f251047a6d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:01.524 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:01.524 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:01.524 7f251047a6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:01.524 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:01.524 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:01.525 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:01.525 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:01.525 7f251047a6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:01.525 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:01.525 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:01.525 7f251047a6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:11:01.525 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:05.527 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:10.533 7f251047a6d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:10.533 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:10.534 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:10.534 7f251047a6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:10.534 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:10.534 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:10.534 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:10.534 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:10.534 7f251047a6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:10.534 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:10.534 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:10.534 7f251047a6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:11:10.535 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:14.537 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:19.542 7f251047a6d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:19.542 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:19.542 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:19.542 7f251047a6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:19.543 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:19.543 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:19.543 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:19.543 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:19.543 7f251047a6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:19.543 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:19.543 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:19.544 7f251047a6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:11:19.544 7f251047a6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:23.546 7f250d57de10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:28.552 7f250d57de10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:28.552 7f250d57de10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:28.552 7f250d57de10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:28.552 7f250d57de10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:28.552 7f250d57de10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:28.553 7f250d57de10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:28.553 7f250d57de10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:28.553 7f250d57de10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:28.553 7f250d57de10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:28.553 7f250d57de10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:28.553 7f250d57de10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:28.553 7f250d57de10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:11:28.553 7f250d57de10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:32.554 7f250cb34978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:37.558 7f250cb34978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:37.558 7f250cb34978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:37.558 7f250cb34978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:37.559 7f250cb34978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:37.559 7f250cb34978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:37.559 7f250cb34978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:37.559 7f250cb34978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:37.559 7f250cb34978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:37.559 7f250cb34978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:37.559 7f250cb34978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:37.559 7f250cb34978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:37.559 7f250cb34978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:11:37.559 7f250cb34978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:41.558 7f250dd7f048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:46.561 7f250dd7f048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:46.562 7f250dd7f048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:46.562 7f250dd7f048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:46.562 7f250dd7f048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:46.562 7f250dd7f048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:46.562 7f250dd7f048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:46.563 7f250dd7f048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:46.563 7f250dd7f048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:46.563 7f250dd7f048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:46.563 7f250dd7f048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:46.563 7f250dd7f048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:46.563 7f250dd7f048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:11:46.563 7f250dd7f048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:50.271 7f250dd7f978 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:11:50.484 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51300300, partition_space_available_1k=29790184, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101848, cpu_share=0.0, mem_res=96624, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84608, cpu_share=0.0, mem_res=78748, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93576, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188848, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=41940, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15166784, free=11044876, buffers=991976, cached=5461484, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.99, five_min_avg=6.44, fifteen_min_avg=4.52, cpu_share=2.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:11:50.484 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:50.485 7f250c9c6780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51300300 partition_space_available_1k = 29790184 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 101848 cpu_share = 0.0 mem_res = 96624 >>contrail-schema : << mem_virt = 84608 cpu_share = 0.0 mem_res = 78748 >>contrail-svc-monitor : << mem_virt = 93576 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 188848 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 41940 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15166784 free = 11044876 buffers = 991976 cached = 5461484 node_type = config-node >> system_cpu_usage = << one_min_avg = 3.99 five_min_avg = 6.44 fifteen_min_avg = 4.52 cpu_share = 2.88 node_type = config-node >> >> 06/02/2025 02:11:50.485 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:50.562 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:11:55.569 7f250c9c6780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:11:55.569 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:11:55.569 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:55.569 7f250c9c6780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:11:55.569 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:11:55.569 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:11:55.570 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:11:55.570 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:55.570 7f250c9c6780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:11:55.570 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:55.570 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:55.570 7f250c9c6780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:11:55.570 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:11:59.572 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:04.577 7f250c9c6780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:04.577 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:04.577 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:04.577 7f250c9c6780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:04.577 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:04.577 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:04.578 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:04.578 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:04.578 7f250c9c6780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:04.578 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:04.578 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:04.578 7f250c9c6780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:12:04.578 7f250c9c6780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:08.580 7f250d57d8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:13.588 7f250d57d8d0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:13.588 7f250d57d8d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:13.588 7f250d57d8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:13.588 7f250d57d8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:13.588 7f250d57d8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:13.588 7f250d57d8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:13.589 7f250d57d8d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:13.589 7f250d57d8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:13.589 7f250d57d8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:13.589 7f250d57d8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:13.589 7f250d57d8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:13.589 7f250d57d8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:12:13.589 7f250d57d8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:17.590 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:22.597 7f250d56f6d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:22.597 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:22.597 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:22.597 7f250d56f6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:22.598 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:22.598 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:22.598 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:22.598 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:22.598 7f250d56f6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:22.598 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:22.598 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:22.599 7f250d56f6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:12:22.599 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:26.600 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:31.602 7f250d56f6d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:31.602 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:31.602 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:31.602 7f250d56f6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:31.603 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:31.603 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:31.603 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:31.603 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:31.603 7f250d56f6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:31.604 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:31.604 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:31.604 7f250d56f6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:12:31.604 7f250d56f6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:35.606 7f250dd7fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:40.612 7f250dd7fa20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:40.612 7f250dd7fa20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:40.612 7f250dd7fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:40.613 7f250dd7fa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:40.613 7f250dd7fa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:40.613 7f250dd7fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:40.613 7f250dd7fa20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:40.613 7f250dd7fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:40.614 7f250dd7fa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:40.614 7f250dd7fa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:40.614 7f250dd7fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:40.614 7f250dd7fa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:12:40.614 7f250dd7fa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:44.613 7f250dd6ceb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:49.616 7f250dd6ceb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:49.616 7f250dd6ceb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:49.616 7f250dd6ceb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:49.617 7f250dd6ceb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:49.617 7f250dd6ceb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:49.617 7f250dd6ceb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:49.617 7f250dd6ceb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:49.617 7f250dd6ceb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:49.617 7f250dd6ceb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:49.618 7f250dd6ceb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:49.618 7f250dd6ceb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:49.618 7f250dd6ceb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:12:49.618 7f250dd6ceb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:50.473 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51313212, partition_space_available_1k=29777272, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101852, cpu_share=0.0, mem_res=96624, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84604, cpu_share=0.0, mem_res=78748, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93572, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188888, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=41960, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15133528, free=9742348, buffers=991988, cached=6797256, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.35, five_min_avg=6.81, fifteen_min_avg=4.77, cpu_share=5.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:12:50.474 7f250dd7fb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:50.474 7f250dd7fb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51313212 partition_space_available_1k = 29777272 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 101852 cpu_share = 0.0 mem_res = 96624 >>contrail-schema : << mem_virt = 84604 cpu_share = 0.0 mem_res = 78748 >>contrail-svc-monitor : << mem_virt = 93572 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 188888 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 41960 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15133528 free = 9742348 buffers = 991988 cached = 6797256 node_type = config-node >> system_cpu_usage = << one_min_avg = 6.35 five_min_avg = 6.81 fifteen_min_avg = 4.77 cpu_share = 5.25 node_type = config-node >> >> 06/02/2025 02:12:50.474 7f250dd7fb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:53.617 7f250dd7fb70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:12:58.622 7f250dd7fb70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:12:58.622 7f250dd7fb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:12:58.622 7f250dd7fb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:58.623 7f250dd7fb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:12:58.623 7f250dd7fb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:12:58.623 7f250dd7fb70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:12:58.623 7f250dd7fb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:12:58.623 7f250dd7fb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:58.623 7f250dd7fb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:12:58.623 7f250dd7fb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:58.623 7f250dd7fb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:12:58.624 7f250dd7fb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:12:58.624 7f250dd7fb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:02.623 7f251047c6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:07.629 7f251047c6d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:07.629 7f251047c6d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:07.629 7f251047c6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:07.630 7f251047c6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:07.630 7f251047c6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:07.630 7f251047c6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:07.630 7f251047c6d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:07.630 7f251047c6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:07.630 7f251047c6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:07.630 7f251047c6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:07.630 7f251047c6d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:07.631 7f251047c6d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:13:07.631 7f251047c6d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:11.633 7f250d56a978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:16.639 7f250d56a978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:16.639 7f250d56a978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:16.640 7f250d56a978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:16.640 7f250d56a978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:16.640 7f250d56a978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:16.640 7f250d56a978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:16.640 7f250d56a978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:16.640 7f250d56a978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:16.640 7f250d56a978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:16.641 7f250d56a978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:16.641 7f250d56a978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:16.641 7f250d56a978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:13:16.641 7f250d56a978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:20.643 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:25.649 7f2510660240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:25.649 7f2510660240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:25.649 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:25.649 7f2510660240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:25.649 7f2510660240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:25.649 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:25.650 7f2510660240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:25.650 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:25.650 7f2510660240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:25.650 7f2510660240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:25.650 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:25.650 7f2510660240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:13:25.650 7f2510660240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:29.653 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:34.658 7f2510660240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:34.658 7f2510660240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:34.658 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:34.658 7f2510660240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:34.658 7f2510660240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:34.658 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:34.659 7f2510660240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:34.659 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:34.659 7f2510660240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:34.659 7f2510660240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:34.659 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:34.659 7f2510660240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:13:34.660 7f2510660240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:38.662 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:43.665 7f2510660240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:43.665 7f2510660240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:43.665 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:43.665 7f2510660240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:43.665 7f2510660240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:43.665 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:43.666 7f2510660240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:43.666 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:43.666 7f2510660240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:43.666 7f2510660240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:43.667 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:43.667 7f2510660240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:13:43.667 7f2510660240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:47.665 7f251047ee10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:13:50.483 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51319852, partition_space_available_1k=29770632, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102096, cpu_share=0.0, mem_res=96800, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84656, cpu_share=0.0, mem_res=78748, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93628, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188896, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42020, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15131316, free=9737164, buffers=991992, cached=6804648, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.5, five_min_avg=6.15, fifteen_min_avg=4.67, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:13:52.668 7f251047ee10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:13:52.669 7f251047ee10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:13:52.669 7f251047ee10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:52.669 7f251047ee10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:13:52.669 7f251047ee10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:52.669 7f251047ee10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:52.669 7f251047ee10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51319852 partition_space_available_1k = 29770632 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102096 cpu_share = 0.0 mem_res = 96800 >>contrail-schema : << mem_virt = 84656 cpu_share = 0.0 mem_res = 78748 >>contrail-svc-monitor : << mem_virt = 93628 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 188896 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42020 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15131316 free = 9737164 buffers = 991992 cached = 6804648 node_type = config-node >> system_cpu_usage = << one_min_avg = 4.5 five_min_avg = 6.15 fifteen_min_avg = 4.67 cpu_share = 2.62 node_type = config-node >> >> 06/02/2025 02:13:52.669 7f251047ee10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:13:52.669 7f251047ee10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:13:52.670 7f251047ee10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:13:52.670 7f251047ee10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:52.670 7f251047ee10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:13:52.670 7f251047ee10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:52.670 7f251047ee10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:52.670 7f251047ee10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:13:52.670 7f251047ee10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:13:56.670 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:01.677 7f251047ad68 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:01.677 7f251047ad68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:01.677 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:01.677 7f251047ad68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:01.677 7f251047ad68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:01.677 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:01.678 7f251047ad68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:01.678 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:01.678 7f251047ad68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:01.678 7f251047ad68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:01.678 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:01.678 7f251047ad68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:14:01.678 7f251047ad68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:05.679 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:10.686 7f251687fe10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:10.686 7f251687fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:10.686 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:10.687 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:10.687 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:10.687 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:10.687 7f251687fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:10.687 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:10.688 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:10.688 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:10.688 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:10.688 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:14:10.688 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:14.690 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:19.694 7f251687fe10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:19.695 7f251687fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:19.695 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:19.695 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:19.695 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:19.695 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:19.695 7f251687fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:19.695 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:19.695 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:19.695 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:19.696 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:19.696 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:14:19.696 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:23.698 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:28.704 7f251687fe10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:28.705 7f251687fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:28.705 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:28.705 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:28.705 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:28.705 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:28.706 7f251687fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:28.706 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:28.706 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:28.706 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:28.706 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:28.706 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:14:28.706 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:32.705 7f2510460828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:37.710 7f2510460828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:37.710 7f2510460828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:37.710 7f2510460828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:37.710 7f2510460828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:37.710 7f2510460828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:37.710 7f2510460828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:37.711 7f2510460828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:37.711 7f2510460828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:37.711 7f2510460828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:37.711 7f2510460828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:37.711 7f2510460828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:37.711 7f2510460828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:14:37.711 7f2510460828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:41.710 7f2510460828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:46.713 7f2510460828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:46.713 7f2510460828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:46.713 7f2510460828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:46.713 7f2510460828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:46.714 7f2510460828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:46.714 7f2510460828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:46.714 7f2510460828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:46.715 7f2510460828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:46.715 7f2510460828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:46.715 7f2510460828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:46.715 7f2510460828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:46.715 7f2510460828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:14:46.715 7f2510460828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:50.478 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51325712, partition_space_available_1k=29764772, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102192, cpu_share=0.0, mem_res=96856, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84688, cpu_share=0.0, mem_res=78752, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93656, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188960, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42004, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15190780, free=9662688, buffers=992000, cached=6819652, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.71, five_min_avg=5.33, fifteen_min_avg=4.48, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:14:50.478 7f251047c390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:50.479 7f251047c390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51325712 partition_space_available_1k = 29764772 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102192 cpu_share = 0.0 mem_res = 96856 >>contrail-schema : << mem_virt = 84688 cpu_share = 0.0 mem_res = 78752 >>contrail-svc-monitor : << mem_virt = 93656 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 188960 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42004 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15190780 free = 9662688 buffers = 992000 cached = 6819652 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.71 five_min_avg = 5.33 fifteen_min_avg = 4.48 cpu_share = 2.75 node_type = config-node >> >> 06/02/2025 02:14:50.479 7f251047c390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:50.713 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:14:55.719 7f250cb22c18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:14:55.720 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:14:55.720 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:55.720 7f250cb22c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:14:55.720 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:14:55.720 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:14:55.720 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:14:55.721 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:55.721 7f250cb22c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:14:55.721 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:55.721 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:55.721 7f250cb22c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:14:55.721 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:14:59.724 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:04.321 7f250dd9dac8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:15:04.726 7f250cb22c18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:04.726 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:04.726 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:04.727 7f250cb22c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:04.727 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:04.727 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:04.727 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:04.727 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:04.727 7f250cb22c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:04.727 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:04.727 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:04.728 7f250cb22c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:15:04.728 7f250cb22c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:08.728 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:13.735 7f25105f52e8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:13.736 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:13.736 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:13.736 7f25105f52e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:13.736 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:13.736 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:13.736 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:13.737 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:13.737 7f25105f52e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:13.737 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:13.737 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:13.737 7f25105f52e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:15:13.737 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:14.645 7f25105f52e8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:15:17.739 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:22.745 7f25105f52e8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:22.745 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:22.745 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:22.745 7f25105f52e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:22.745 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:22.746 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:22.746 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:22.746 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:22.746 7f25105f52e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:22.746 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:22.746 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:22.746 7f25105f52e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:15:22.746 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:25.390 7f25105f52e8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:15:26.745 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:31.751 7f25105f52e8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:31.752 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:31.752 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:31.752 7f25105f52e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:31.752 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:31.752 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:31.753 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:31.753 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:31.753 7f25105f52e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:31.753 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:31.753 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:31.753 7f25105f52e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:15:31.753 7f25105f52e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:35.753 7f251044f048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:40.760 7f251044f048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:40.760 7f251044f048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:40.760 7f251044f048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:40.760 7f251044f048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:40.760 7f251044f048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:40.760 7f251044f048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:40.761 7f251044f048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:40.761 7f251044f048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:40.761 7f251044f048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:40.761 7f251044f048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:40.761 7f251044f048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:40.761 7f251044f048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:15:40.761 7f251044f048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:44.761 7f250dd6ccc0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:49.764 7f250dd6ccc0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:49.765 7f250dd6ccc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:49.765 7f250dd6ccc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:49.765 7f250dd6ccc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:49.765 7f250dd6ccc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:49.765 7f250dd6ccc0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:49.765 7f250dd6ccc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:49.766 7f250dd6ccc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:49.766 7f250dd6ccc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:49.766 7f250dd6ccc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:49.766 7f250dd6ccc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:49.766 7f250dd6ccc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:15:49.766 7f250dd6ccc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:50.486 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51335104, partition_space_available_1k=29755380, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102208, cpu_share=0.0, mem_res=96856, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84692, cpu_share=0.0, mem_res=78760, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93652, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188996, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42032, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15157116, free=9685336, buffers=993360, cached=6829308, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.91, five_min_avg=4.92, fifteen_min_avg=4.39, cpu_share=3.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:15:50.486 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:50.487 7f25162a7240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51335104 partition_space_available_1k = 29755380 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102208 cpu_share = 0.0 mem_res = 96856 >>contrail-schema : << mem_virt = 84692 cpu_share = 0.0 mem_res = 78760 >>contrail-svc-monitor : << mem_virt = 93652 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 188996 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42032 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15157116 free = 9685336 buffers = 993360 cached = 6829308 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.91 five_min_avg = 4.92 fifteen_min_avg = 4.39 cpu_share = 3.38 node_type = config-node >> >> 06/02/2025 02:15:50.487 7f25162a7240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:53.768 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:15:58.774 7f25162a7240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:15:58.774 7f25162a7240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:15:58.774 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:58.775 7f25162a7240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:15:58.775 7f25162a7240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:15:58.775 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:15:58.775 7f25162a7240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:15:58.775 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:58.775 7f25162a7240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:15:58.776 7f25162a7240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:58.776 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:15:58.776 7f25162a7240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:15:58.776 7f25162a7240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:02.778 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:07.781 7f25162a7240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:07.781 7f25162a7240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:07.781 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:07.781 7f25162a7240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:07.781 7f25162a7240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:07.781 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:07.782 7f25162a7240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:07.782 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:07.782 7f25162a7240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:07.782 7f25162a7240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:07.782 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:07.782 7f25162a7240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:16:07.782 7f25162a7240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:11.785 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:16.790 7f25162a7240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:16.790 7f25162a7240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:16.791 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:16.791 7f25162a7240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:16.791 7f25162a7240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:16.791 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:16.791 7f25162a7240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:16.791 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:16.791 7f25162a7240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:16.791 7f25162a7240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:16.791 7f25162a7240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:16.792 7f25162a7240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:16:16.792 7f25162a7240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:20.793 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:25.801 7f2510606240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:25.801 7f2510606240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:25.801 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:25.801 7f2510606240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:25.801 7f2510606240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:25.801 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:25.802 7f2510606240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:25.802 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:25.802 7f2510606240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:25.802 7f2510606240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:25.802 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:25.802 7f2510606240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:16:25.802 7f2510606240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:29.806 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:34.810 7f2510606240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:34.810 7f2510606240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:34.810 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:34.811 7f2510606240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:34.811 7f2510606240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:34.811 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:34.813 7f2510606240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:34.813 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:34.813 7f2510606240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:34.813 7f2510606240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:34.813 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:34.813 7f2510606240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:16:34.813 7f2510606240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:38.813 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:43.816 7f2510606240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:43.817 7f2510606240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:43.817 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:43.817 7f2510606240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:43.817 7f2510606240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:43.817 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:43.818 7f2510606240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:43.818 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:43.819 7f2510606240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:43.819 7f2510606240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:43.819 7f2510606240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:43.819 7f2510606240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:16:43.819 7f2510606240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:47.817 7f251047ea20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:16:49.022 7f250dd8f2e8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:16:50.511 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51346232, partition_space_available_1k=29744252, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102272, cpu_share=0.0, mem_res=96856, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84752, cpu_share=0.0, mem_res=78760, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93708, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189004, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42088, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15247480, free=9559352, buffers=994020, cached=6864268, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.5, five_min_avg=5.44, fifteen_min_avg=4.59, cpu_share=5.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:16:52.821 7f251047ea20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:16:52.821 7f251047ea20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:16:52.821 7f251047ea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:52.822 7f251047ea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:16:52.822 7f251047ea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:52.822 7f251047ea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:52.822 7f251047ea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51346232 partition_space_available_1k = 29744252 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102272 cpu_share = 0.0 mem_res = 96856 >>contrail-schema : << mem_virt = 84752 cpu_share = 0.0 mem_res = 78760 >>contrail-svc-monitor : << mem_virt = 93708 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189004 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42088 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15247480 free = 9559352 buffers = 994020 cached = 6864268 node_type = config-node >> system_cpu_usage = << one_min_avg = 6.5 five_min_avg = 5.44 fifteen_min_avg = 4.59 cpu_share = 5.25 node_type = config-node >> >> 06/02/2025 02:16:52.822 7f251047ea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:16:52.822 7f251047ea20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:16:52.823 7f251047ea20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:16:52.823 7f251047ea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:52.823 7f251047ea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:16:52.823 7f251047ea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:52.823 7f251047ea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:52.823 7f251047ea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:16:52.823 7f251047ea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:16:56.825 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:01.829 7f2511b3d780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:01.829 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:01.829 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:01.829 7f2511b3d780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:01.829 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:01.829 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:01.830 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:01.830 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:01.830 7f2511b3d780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:01.830 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:01.830 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:01.830 7f2511b3d780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:17:01.830 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:05.831 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:10.836 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:10.836 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:10.837 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:10.837 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:10.837 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:10.837 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:10.837 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:10.838 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:10.838 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:10.838 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:10.838 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:10.838 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:17:10.838 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:14.838 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:19.840 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:19.841 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:19.841 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:19.841 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:19.841 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:19.841 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:19.842 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:19.842 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:19.842 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:19.842 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:19.842 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:19.843 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:17:19.843 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:23.843 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:28.849 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:28.849 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:28.849 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:28.849 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:28.849 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:28.849 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:28.850 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:28.850 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:28.850 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:28.850 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:28.850 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:28.850 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:17:28.850 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:32.853 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:37.857 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:37.857 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:37.857 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:37.857 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:37.857 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:37.857 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:37.858 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:37.858 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:37.858 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:37.858 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:37.858 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:37.858 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:17:37.858 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:41.858 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:46.862 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:46.863 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:46.863 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:46.863 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:46.863 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:46.864 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:46.865 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:46.865 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:46.865 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:46.865 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:46.865 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:46.865 7f2511ae9438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:17:46.867 7f2511ae9438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:50.492 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51354888, partition_space_available_1k=29735596, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102284, cpu_share=0.0, mem_res=96856, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84776, cpu_share=0.0, mem_res=78764, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93736, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189068, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42076, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15208228, free=9576512, buffers=994116, cached=6886264, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.66, five_min_avg=4.79, fifteen_min_avg=4.42, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:17:50.492 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:50.492 7f250d592198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51354888 partition_space_available_1k = 29735596 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102284 cpu_share = 0.0 mem_res = 96856 >>contrail-schema : << mem_virt = 84776 cpu_share = 0.0 mem_res = 78764 >>contrail-svc-monitor : << mem_virt = 93736 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189068 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42076 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15208228 free = 9576512 buffers = 994116 cached = 6886264 node_type = config-node >> system_cpu_usage = << one_min_avg = 3.66 five_min_avg = 4.79 fifteen_min_avg = 4.42 cpu_share = 2.62 node_type = config-node >> >> 06/02/2025 02:17:50.492 7f250d592198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:50.862 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:17:55.865 7f250d592198 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:17:55.866 7f250d592198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:17:55.866 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:55.866 7f250d592198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:17:55.866 7f250d592198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:17:55.866 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:17:55.867 7f250d592198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:17:55.867 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:55.867 7f250d592198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:17:55.867 7f250d592198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:55.867 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:55.867 7f250d592198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:17:55.867 7f250d592198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:17:59.870 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:04.872 7f250d592198 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:04.872 7f250d592198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:04.873 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:04.873 7f250d592198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:04.873 7f250d592198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:04.873 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:04.873 7f250d592198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:04.873 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:04.874 7f250d592198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:04.874 7f250d592198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:04.874 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:04.874 7f250d592198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:18:04.874 7f250d592198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:08.873 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:13.877 7f251687fe10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:13.878 7f251687fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:13.878 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:13.878 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:13.878 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:13.878 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:13.878 7f251687fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:13.879 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:13.879 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:13.879 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:13.879 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:13.879 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:18:13.879 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:17.880 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:22.883 7f251687fe10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:22.883 7f251687fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:22.883 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:22.884 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:22.884 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:22.884 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:22.884 7f251687fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:22.884 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:22.884 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:22.884 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:22.885 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:22.885 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:18:22.885 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:26.885 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:31.891 7f251687fe10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:31.891 7f251687fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:31.891 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:31.892 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:31.892 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:31.892 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:31.892 7f251687fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:31.892 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:31.892 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:31.893 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:31.893 7f251687fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:31.893 7f251687fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:18:31.893 7f251687fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:35.893 7f2510475390 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:40.899 7f2510475390 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:40.899 7f2510475390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:40.899 7f2510475390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:40.899 7f2510475390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:40.899 7f2510475390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:40.899 7f2510475390 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:40.900 7f2510475390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:40.900 7f2510475390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:40.900 7f2510475390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:40.900 7f2510475390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:40.900 7f2510475390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:40.900 7f2510475390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:18:40.900 7f2510475390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:44.900 7f250dd73588 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:49.903 7f250dd73588 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:49.905 7f250dd73588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:49.905 7f250dd73588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:49.905 7f250dd73588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:49.905 7f250dd73588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:49.905 7f250dd73588 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:49.906 7f250dd73588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:49.906 7f250dd73588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:49.906 7f250dd73588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:49.906 7f250dd73588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:49.906 7f250dd73588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:49.907 7f250dd73588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:18:49.907 7f250dd73588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:50.490 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51360416, partition_space_available_1k=29730068, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102316, cpu_share=0.0, mem_res=96856, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84772, cpu_share=0.0, mem_res=78764, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93740, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189108, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42148, cpu_share=0.01, mem_res=39664, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15196932, free=9581008, buffers=994116, cached=6893064, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.52, five_min_avg=4.26, fifteen_min_avg=4.26, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:18:50.491 7f250d56a048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:50.491 7f250d56a048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51360416 partition_space_available_1k = 29730068 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102316 cpu_share = 0.0 mem_res = 96856 >>contrail-schema : << mem_virt = 84772 cpu_share = 0.0 mem_res = 78764 >>contrail-svc-monitor : << mem_virt = 93740 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189108 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42148 cpu_share = 0.01 mem_res = 39664 >> } system_mem_usage = << total = 32665120 used = 15196932 free = 9581008 buffers = 994116 cached = 6893064 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.52 five_min_avg = 4.26 fifteen_min_avg = 4.26 cpu_share = 2.62 node_type = config-node >> >> 06/02/2025 02:18:50.491 7f250d56a048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:53.907 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:18:58.910 7f2510606a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:18:58.911 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:18:58.911 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:58.911 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:18:58.911 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:18:58.911 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:18:58.911 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:18:58.911 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:58.912 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:18:58.912 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:58.912 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:18:58.912 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:18:58.912 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:02.915 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:07.917 7f2510606a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:07.917 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:07.917 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:07.918 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:07.918 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:07.918 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:07.918 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:07.918 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:07.918 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:07.918 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:07.918 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:07.918 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:19:07.918 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:11.922 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:16.928 7f2510606a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:16.928 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:16.928 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:16.928 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:16.928 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:16.928 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:16.929 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:16.929 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:16.929 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:16.929 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:16.929 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:16.929 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:19:16.929 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:20.929 7f25106068d0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:25.934 7f25106068d0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:25.934 7f25106068d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:25.935 7f25106068d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:25.936 7f25106068d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:25.937 7f25106068d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:25.937 7f25106068d0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:25.937 7f25106068d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:25.937 7f25106068d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:25.938 7f25106068d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:25.938 7f25106068d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:25.938 7f25106068d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:25.938 7f25106068d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:19:25.938 7f25106068d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:29.935 7f25106068d0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:34.942 7f25106068d0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:34.942 7f25106068d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:34.942 7f25106068d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:34.942 7f25106068d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:34.942 7f25106068d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:34.942 7f25106068d0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:34.943 7f25106068d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:34.943 7f25106068d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:34.943 7f25106068d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:34.943 7f25106068d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:34.943 7f25106068d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:34.943 7f25106068d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:19:34.943 7f25106068d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:38.945 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:43.948 7f250d5920f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:43.948 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:43.949 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:43.949 7f250d5920f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:43.949 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:43.949 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:43.950 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:43.950 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:43.950 7f250d5920f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:43.950 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:43.950 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:43.950 7f250d5920f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:19:43.950 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:47.950 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:19:50.496 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51365776, partition_space_available_1k=29724708, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102388, cpu_share=0.0, mem_res=96856, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84840, cpu_share=0.0, mem_res=78780, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93792, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189112, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42216, cpu_share=0.01, mem_res=39664, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15185644, free=9586212, buffers=994124, cached=6899140, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.33, five_min_avg=3.91, fifteen_min_avg=4.14, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:19:52.952 7f250d5920f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:19:52.953 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:19:52.953 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:52.953 7f250d5920f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:19:52.953 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:52.953 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:52.953 7f250d5920f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51365776 partition_space_available_1k = 29724708 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102388 cpu_share = 0.0 mem_res = 96856 >>contrail-schema : << mem_virt = 84840 cpu_share = 0.0 mem_res = 78780 >>contrail-svc-monitor : << mem_virt = 93792 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189112 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42216 cpu_share = 0.01 mem_res = 39664 >> } system_mem_usage = << total = 32665120 used = 15185644 free = 9586212 buffers = 994124 cached = 6899140 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.33 five_min_avg = 3.91 fifteen_min_avg = 4.14 cpu_share = 2.62 node_type = config-node >> >> 06/02/2025 02:19:52.953 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:19:52.954 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:19:52.954 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:19:52.954 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:52.954 7f250d5920f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:19:52.954 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:52.954 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:52.954 7f250d5920f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:19:52.954 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:19:56.956 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:01.962 7f251047ad68 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:01.963 7f251047ad68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:01.963 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:01.963 7f251047ad68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:01.963 7f251047ad68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:01.963 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:01.964 7f251047ad68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:01.964 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:01.964 7f251047ad68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:01.964 7f251047ad68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:01.964 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:01.964 7f251047ad68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:20:01.964 7f251047ad68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:05.963 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:10.969 7f251047ab70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:10.969 7f251047ab70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:10.969 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:10.969 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:10.969 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:10.969 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:10.970 7f251047ab70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:10.970 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:10.970 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:10.970 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:10.970 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:10.970 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:20:10.970 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:14.973 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:16.723 7f250d58f048 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:20:19.978 7f251047ab70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:19.978 7f251047ab70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:19.978 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:19.978 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:19.978 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:19.978 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:19.979 7f251047ab70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:19.979 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:19.981 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:19.981 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:19.981 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:19.981 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:20:19.981 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:23.981 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:27.436 7f250d56f2e8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:20:28.984 7f250d592198 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:28.984 7f250d592198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:28.984 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:28.984 7f250d592198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:28.984 7f250d592198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:28.985 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:28.985 7f250d592198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:28.985 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:28.985 7f250d592198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:28.985 7f250d592198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:28.985 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:28.986 7f250d592198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:20:28.986 7f250d592198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:32.984 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:37.873 7f250d5922e8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:20:37.986 7f250d592198 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:37.986 7f250d592198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:37.986 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:37.986 7f250d592198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:37.986 7f250d592198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:37.986 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:37.987 7f250d592198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:37.987 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:37.987 7f250d592198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:37.987 7f250d592198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:37.987 7f250d592198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:37.987 7f250d592198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:20:37.987 7f250d592198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:41.987 7f250d56fc18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:46.991 7f250d56fc18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:46.991 7f250d56fc18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:46.991 7f250d56fc18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:46.991 7f250d56fc18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:46.991 7f250d56fc18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:46.991 7f250d56fc18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:46.992 7f250d56fc18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:46.992 7f250d56fc18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:46.992 7f250d56fc18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:46.992 7f250d56fc18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:46.992 7f250d56fc18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:46.992 7f250d56fc18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:20:46.992 7f250d56fc18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:50.501 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51379740, partition_space_available_1k=29710744, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102416, cpu_share=0.0, mem_res=96876, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84880, cpu_share=0.0, mem_res=78788, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93820, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189180, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42148, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15249088, free=9509780, buffers=995500, cached=6910752, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.31, five_min_avg=3.61, fifteen_min_avg=4.02, cpu_share=3.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:20:50.501 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:50.501 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51379740 partition_space_available_1k = 29710744 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102416 cpu_share = 0.0 mem_res = 96876 >>contrail-schema : << mem_virt = 84880 cpu_share = 0.0 mem_res = 78788 >>contrail-svc-monitor : << mem_virt = 93820 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189180 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42148 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15249088 free = 9509780 buffers = 995500 cached = 6910752 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.31 five_min_avg = 3.61 fifteen_min_avg = 4.02 cpu_share = 3.12 node_type = config-node >> >> 06/02/2025 02:20:50.501 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:50.992 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:20:55.998 7f251009f588 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:20:55.999 7f251009f588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:20:55.999 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:55.999 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:20:55.999 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:20:55.999 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:20:55.999 7f251009f588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:20:55.999 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:55.999 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:20:56.000 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:56.000 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:20:56.000 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:20:56.000 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:00.003 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:05.009 7f251009f588 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:05.009 7f251009f588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:05.010 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:05.010 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:05.010 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:05.010 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:05.010 7f251009f588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:05.011 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:05.011 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:05.011 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:05.011 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:05.011 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:21:05.011 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:09.010 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:14.017 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:14.017 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:14.017 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:14.017 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:14.017 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:14.017 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:14.018 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:14.018 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:14.018 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:14.018 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:14.018 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:14.018 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:21:14.018 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:18.020 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:23.025 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:23.025 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:23.025 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:23.025 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:23.025 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:23.025 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:23.026 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:23.026 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:23.026 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:23.026 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:23.026 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:23.026 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:21:23.026 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:27.033 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:32.049 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:32.049 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:32.049 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:32.049 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:32.049 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:32.049 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:32.050 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:32.050 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:32.050 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:32.050 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:32.050 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:32.051 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:21:32.051 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:36.053 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:41.058 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:41.058 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:41.058 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:41.059 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:41.059 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:41.059 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:41.059 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:41.059 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:41.059 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:41.059 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:41.059 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:41.060 7f250cb34c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:21:41.060 7f250cb34c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:45.059 7f250dd8f630 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:46.415 7f251048a0f0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:21:50.063 7f250dd8f630 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:50.063 7f250dd8f630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:50.063 7f250dd8f630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:50.063 7f250dd8f630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:50.063 7f250dd8f630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:50.063 7f250dd8f630 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:50.064 7f250dd8f630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:50.064 7f250dd8f630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:50.064 7f250dd8f630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:50.064 7f250dd8f630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:50.064 7f250dd8f630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:50.064 7f250dd8f630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:21:50.064 7f250dd8f630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:50.506 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51381780, partition_space_available_1k=29708704, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102568, cpu_share=0.0, mem_res=96988, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84872, cpu_share=0.0, mem_res=78788, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93820, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189216, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42172, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15335008, free=9401044, buffers=996088, cached=6932980, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.25, five_min_avg=4.46, fifteen_min_avg=4.28, cpu_share=5.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:21:50.507 7f2510660240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:50.507 7f2510660240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51381780 partition_space_available_1k = 29708704 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102568 cpu_share = 0.0 mem_res = 96988 >>contrail-schema : << mem_virt = 84872 cpu_share = 0.0 mem_res = 78788 >>contrail-svc-monitor : << mem_virt = 93820 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189216 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42172 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15335008 free = 9401044 buffers = 996088 cached = 6932980 node_type = config-node >> system_cpu_usage = << one_min_avg = 6.25 five_min_avg = 4.46 fifteen_min_avg = 4.28 cpu_share = 5.12 node_type = config-node >> >> 06/02/2025 02:21:50.507 7f2510660240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:54.065 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:21:59.071 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:21:59.072 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:21:59.072 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:59.072 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:21:59.072 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:21:59.072 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:21:59.072 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:21:59.072 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:59.073 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:21:59.073 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:59.073 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:21:59.073 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:21:59.073 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:03.074 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:08.076 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:08.076 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:08.077 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:08.077 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:08.077 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:08.077 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:08.078 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:08.078 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:08.078 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:08.078 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:08.078 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:08.078 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:22:08.078 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:12.080 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:17.085 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:17.085 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:17.085 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:17.085 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:17.085 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:17.086 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:17.086 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:17.086 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:17.086 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:17.086 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:17.086 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:17.087 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:22:17.087 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:21.088 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:26.094 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:26.094 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:26.094 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:26.094 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:26.094 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:26.094 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:26.095 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:26.095 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:26.095 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:26.095 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:26.095 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:26.095 7f250c9c6f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:22:26.095 7f250c9c6f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:30.098 7f250d57d780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:35.104 7f250d57d780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:35.105 7f250d57d780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:35.105 7f250d57d780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:35.105 7f250d57d780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:35.105 7f250d57d780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:35.105 7f250d57d780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:35.106 7f250d57d780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:35.106 7f250d57d780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:35.106 7f250d57d780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:35.106 7f250d57d780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:35.106 7f250d57d780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:35.106 7f250d57d780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:22:35.106 7f250d57d780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:39.106 7f250d56fcc0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:44.110 7f250d56fcc0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:44.111 7f250d56fcc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:44.111 7f250d56fcc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:44.111 7f250d56fcc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:44.112 7f250d56fcc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:44.112 7f250d56fcc0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:44.113 7f250d56fcc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:44.113 7f250d56fcc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:44.113 7f250d56fcc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:44.113 7f250d56fcc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:44.113 7f250d56fcc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:44.114 7f250d56fcc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:22:44.114 7f250d56fcc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:48.112 7f250d56acc0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:22:50.497 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51390388, partition_space_available_1k=29700096, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102744, cpu_share=0.0, mem_res=97100, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84928, cpu_share=0.0, mem_res=78788, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93876, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189224, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42228, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15258736, free=9471520, buffers=996100, cached=6938764, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.69, five_min_avg=4.06, fifteen_min_avg=4.15, cpu_share=2.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:22:53.114 7f250d56acc0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:22:53.114 7f250d56acc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:22:53.115 7f250d56acc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:53.115 7f250d56acc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:22:53.115 7f250d56acc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:53.115 7f250d56acc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:53.115 7f250d56acc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51390388 partition_space_available_1k = 29700096 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102744 cpu_share = 0.0 mem_res = 97100 >>contrail-schema : << mem_virt = 84928 cpu_share = 0.0 mem_res = 78788 >>contrail-svc-monitor : << mem_virt = 93876 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189224 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42228 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15258736 free = 9471520 buffers = 996100 cached = 6938764 node_type = config-node >> system_cpu_usage = << one_min_avg = 3.69 five_min_avg = 4.06 fifteen_min_avg = 4.15 cpu_share = 2.38 node_type = config-node >> >> 06/02/2025 02:22:53.115 7f250d56acc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:22:53.115 7f250d56acc0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:22:53.116 7f250d56acc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:22:53.116 7f250d56acc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:53.116 7f250d56acc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:22:53.116 7f250d56acc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:53.116 7f250d56acc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:53.116 7f250d56acc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:22:53.116 7f250d56acc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:22:57.118 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:02.124 7f250d5806d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:02.124 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:02.124 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:02.124 7f250d5806d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:02.124 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:02.124 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:02.125 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:02.125 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:02.125 7f250d5806d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:02.125 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:02.125 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:02.125 7f250d5806d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:23:02.125 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:06.127 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:11.133 7f250d5806d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:11.133 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:11.133 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:11.133 7f250d5806d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:11.133 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:11.133 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:11.134 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:11.134 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:11.134 7f250d5806d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:11.134 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:11.134 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:11.134 7f250d5806d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:23:11.134 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:15.137 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:20.143 7f250d5806d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:20.144 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:20.144 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:20.144 7f250d5806d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:20.144 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:20.144 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:20.144 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:20.144 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:20.144 7f250d5806d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:20.145 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:20.145 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:20.145 7f250d5806d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:23:20.145 7f250d5806d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:24.144 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:29.151 7f25100f8978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:29.152 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:29.152 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:29.152 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:29.152 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:29.152 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:29.152 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:29.154 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:29.154 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:29.154 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:29.155 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:29.155 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:23:29.155 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:33.156 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:38.158 7f25100f8978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:38.158 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:38.158 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:38.158 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:38.158 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:38.159 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:38.159 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:38.159 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:38.160 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:38.160 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:38.160 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:38.160 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:23:38.160 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:42.159 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:47.162 7f25100f8978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:47.162 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:47.162 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:47.162 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:47.162 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:47.162 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:47.163 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:47.163 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:47.163 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:47.163 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:47.163 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:47.164 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:23:47.164 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:50.506 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51395820, partition_space_available_1k=29694664, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102752, cpu_share=0.0, mem_res=97100, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84836, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93900, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189284, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42220, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15274456, free=9454668, buffers=996100, cached=6939896, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.4, five_min_avg=3.86, fifteen_min_avg=4.08, cpu_share=2.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:23:50.507 7f251046eb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:50.507 7f251046eb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51395820 partition_space_available_1k = 29694664 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102752 cpu_share = 0.0 mem_res = 97100 >>contrail-schema : << mem_virt = 84836 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 93900 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189284 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42220 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15274456 free = 9454668 buffers = 996100 cached = 6939896 node_type = config-node >> system_cpu_usage = << one_min_avg = 3.4 five_min_avg = 3.86 fifteen_min_avg = 4.08 cpu_share = 2.25 node_type = config-node >> >> 06/02/2025 02:23:50.507 7f251046eb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:51.163 7f251046eb70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:23:56.168 7f251046eb70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:23:56.168 7f251046eb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:23:56.168 7f251046eb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:56.169 7f251046eb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:23:56.169 7f251046eb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:23:56.169 7f251046eb70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:23:56.170 7f251046eb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:23:56.170 7f251046eb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:56.170 7f251046eb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:23:56.170 7f251046eb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:56.171 7f251046eb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:23:56.171 7f251046eb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:23:56.171 7f251046eb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:00.172 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:05.178 7f250e2126d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:05.179 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:05.179 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:05.179 7f250e2126d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:05.179 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:05.179 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:05.180 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:05.180 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:05.180 7f250e2126d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:05.180 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:05.180 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:05.180 7f250e2126d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:24:05.180 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:09.179 7f250cb38630 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:14.186 7f250cb38630 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:14.186 7f250cb38630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:14.186 7f250cb38630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:14.186 7f250cb38630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:14.186 7f250cb38630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:14.186 7f250cb38630 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:14.187 7f250cb38630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:14.187 7f250cb38630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:14.187 7f250cb38630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:14.187 7f250cb38630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:14.187 7f250cb38630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:14.187 7f250cb38630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:24:14.187 7f250cb38630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:18.189 7f250cb38630 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:23.193 7f250cb38630 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:23.193 7f250cb38630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:23.193 7f250cb38630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:23.194 7f250cb38630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:23.194 7f250cb38630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:23.194 7f250cb38630 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:23.194 7f250cb38630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:23.194 7f250cb38630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:23.194 7f250cb38630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:23.194 7f250cb38630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:23.194 7f250cb38630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:23.195 7f250cb38630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:24:23.195 7f250cb38630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:27.197 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:32.201 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:32.201 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:32.201 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:32.202 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:32.202 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:32.202 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:32.202 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:32.202 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:32.202 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:32.202 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:32.202 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:32.203 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:24:32.203 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:36.205 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:41.209 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:41.210 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:41.210 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:41.210 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:41.210 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:41.210 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:41.210 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:41.210 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:41.211 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:41.211 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:41.211 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:41.211 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:24:41.211 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:45.210 7f2510475e10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:50.215 7f2510475e10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:50.215 7f2510475e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:50.215 7f2510475e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:50.215 7f2510475e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:50.215 7f2510475e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:50.215 7f2510475e10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:50.216 7f2510475e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:50.216 7f2510475e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:50.217 7f2510475e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:50.217 7f2510475e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:50.217 7f2510475e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:50.217 7f2510475e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:24:50.217 7f2510475e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:50.508 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51396264, partition_space_available_1k=29694220, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102796, cpu_share=0.0, mem_res=97100, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84832, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93896, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189324, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42244, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15257316, free=9465480, buffers=996100, cached=6946224, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.13, five_min_avg=3.4, fifteen_min_avg=3.9, cpu_share=2.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:24:50.508 7f251047ab70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:50.508 7f251047ab70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51396264 partition_space_available_1k = 29694220 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102796 cpu_share = 0.0 mem_res = 97100 >>contrail-schema : << mem_virt = 84832 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 93896 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189324 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42244 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15257316 free = 9465480 buffers = 996100 cached = 6946224 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.13 five_min_avg = 3.4 fifteen_min_avg = 3.9 cpu_share = 2.25 node_type = config-node >> >> 06/02/2025 02:24:50.508 7f251047ab70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:54.215 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:24:59.221 7f251170d630 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:24:59.221 7f251170d630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:24:59.221 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:59.221 7f251170d630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:24:59.222 7f251170d630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:24:59.222 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:24:59.222 7f251170d630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:24:59.222 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:59.222 7f251170d630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:24:59.222 7f251170d630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:59.222 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:24:59.222 7f251170d630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:24:59.223 7f251170d630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:03.225 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:08.230 7f251170d630 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:08.230 7f251170d630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:08.230 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:08.230 7f251170d630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:08.230 7f251170d630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:08.230 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:08.231 7f251170d630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:08.231 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:08.231 7f251170d630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:08.231 7f251170d630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:08.231 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:08.231 7f251170d630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:25:08.231 7f251170d630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:12.233 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:17.239 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:17.240 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:17.240 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:17.240 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:17.240 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:17.240 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:17.240 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:17.240 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:17.240 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:17.241 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:17.241 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:17.241 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:25:17.241 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:21.243 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:26.248 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:26.248 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:26.248 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:26.248 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:26.249 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:26.249 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:26.249 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:26.249 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:26.249 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:26.249 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:26.250 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:26.250 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:25:26.250 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:30.251 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:35.257 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:35.257 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:35.258 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:35.258 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:35.258 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:35.258 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:35.260 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:35.261 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:35.261 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:35.261 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:35.261 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:35.261 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:25:35.261 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:39.263 7f250d592c18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:44.321 7f250d592c18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:44.321 7f250d592c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:44.321 7f250d592c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:44.321 7f250d592c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:44.322 7f250d592c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:44.322 7f250d592c18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:44.322 7f250d592c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:44.322 7f250d592c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:44.322 7f250d592c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:44.322 7f250d592c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:44.322 7f250d592c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:44.322 7f250d592c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:25:44.322 7f250d592c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:48.323 7f251044f2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:25:50.543 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51398960, partition_space_available_1k=29691524, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102860, cpu_share=0.0, mem_res=97100, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84888, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93952, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189328, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42288, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15368528, free=9340092, buffers=996100, cached=6960400, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.48, five_min_avg=3.93, fifteen_min_avg=4.04, cpu_share=4.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:25:53.326 7f251044f2e8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:25:53.326 7f251044f2e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:25:53.326 7f251044f2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:53.327 7f251044f2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:25:53.327 7f251044f2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:53.327 7f251044f2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:53.327 7f251044f2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51398960 partition_space_available_1k = 29691524 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102860 cpu_share = 0.0 mem_res = 97100 >>contrail-schema : << mem_virt = 84888 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 93952 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189328 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42288 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15368528 free = 9340092 buffers = 996100 cached = 6960400 node_type = config-node >> system_cpu_usage = << one_min_avg = 5.48 five_min_avg = 3.93 fifteen_min_avg = 4.04 cpu_share = 4.62 node_type = config-node >> >> 06/02/2025 02:25:53.327 7f251044f2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:25:53.327 7f251044f2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:25:53.328 7f251044f2e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:25:53.328 7f251044f2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:53.328 7f251044f2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:25:53.328 7f251044f2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:53.328 7f251044f2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:53.328 7f251044f2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:25:53.329 7f251044f2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:25:57.329 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:02.335 7f2510606a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:02.335 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:02.335 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:02.336 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:02.336 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:02.336 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:02.336 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:02.336 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:02.337 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:02.337 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:02.337 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:02.337 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:26:02.337 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:06.338 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:08.685 7f250cb38f60 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:26:11.340 7f2510606a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:11.341 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:11.341 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:11.341 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:11.341 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:11.341 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:11.342 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:11.342 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:11.342 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:11.342 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:11.342 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:11.343 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:26:11.343 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:15.345 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:18.973 7f251048ae10 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:26:20.348 7f2510606a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:20.348 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:20.348 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:20.349 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:20.349 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:20.349 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:20.349 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:20.349 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:20.349 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:20.349 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:20.349 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:20.350 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:26:20.350 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:24.352 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:29.358 7f2510606a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:29.358 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:29.358 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:29.358 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:29.358 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:29.358 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:29.359 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:29.359 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:29.359 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:29.359 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:29.359 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:29.359 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:26:29.359 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:29.514 7f250d57a048 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:26:33.361 7f250d57a048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:38.366 7f250d57a048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:38.367 7f250d57a048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:38.367 7f250d57a048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:38.367 7f250d57a048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:38.367 7f250d57a048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:38.367 7f250d57a048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:38.368 7f250d57a048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:38.368 7f250d57a048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:38.368 7f250d57a048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:38.368 7f250d57a048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:38.368 7f250d57a048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:38.368 7f250d57a048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:26:38.368 7f250d57a048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:42.367 7f251047c780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:47.370 7f251047c780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:47.371 7f251047c780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:47.371 7f251047c780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:47.371 7f251047c780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:47.371 7f251047c780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:47.371 7f251047c780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:47.371 7f251047c780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:47.372 7f251047c780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:47.372 7f251047c780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:47.372 7f251047c780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:47.372 7f251047c780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:47.372 7f251047c780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:26:47.372 7f251047c780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:50.517 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51408880, partition_space_available_1k=29681604, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102864, cpu_share=0.0, mem_res=97100, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84916, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93976, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189392, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42292, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15304816, free=9485224, buffers=996144, cached=6878936, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.28, five_min_avg=3.91, fifteen_min_avg=4.03, cpu_share=3.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:26:50.517 7f251047c828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:50.518 7f251047c828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51408880 partition_space_available_1k = 29681604 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102864 cpu_share = 0.0 mem_res = 97100 >>contrail-schema : << mem_virt = 84916 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 93976 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189392 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42292 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15304816 free = 9485224 buffers = 996144 cached = 6878936 node_type = config-node >> system_cpu_usage = << one_min_avg = 4.28 five_min_avg = 3.91 fifteen_min_avg = 4.03 cpu_share = 3.12 node_type = config-node >> >> 06/02/2025 02:26:50.518 7f251047c828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:51.371 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:26:56.373 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:26:56.373 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:26:56.373 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:56.374 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:26:56.374 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:26:56.374 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:26:56.374 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:26:56.374 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:56.375 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:26:56.375 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:56.375 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:26:56.375 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:26:56.375 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:00.377 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:05.383 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:05.383 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:05.383 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:05.384 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:05.384 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:05.384 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:05.384 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:05.385 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:05.385 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:05.385 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:05.385 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:05.385 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:27:05.385 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:09.387 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:14.391 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:14.391 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:14.391 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:14.392 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:14.392 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:14.392 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:14.392 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:14.393 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:14.393 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:14.393 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:14.393 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:14.393 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:27:14.393 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:18.395 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:23.402 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:23.402 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:23.403 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:23.403 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:23.403 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:23.403 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:23.403 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:23.404 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:23.404 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:23.404 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:23.404 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:23.404 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:27:23.404 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:27.404 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:32.411 7f251047ad68 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:32.411 7f251047ad68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:32.412 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:32.412 7f251047ad68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:32.412 7f251047ad68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:32.412 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:32.413 7f251047ad68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:32.413 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:32.413 7f251047ad68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:32.413 7f251047ad68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:32.413 7f251047ad68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:32.413 7f251047ad68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:27:32.413 7f251047ad68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:33.809 7f251047ad68 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:27:36.413 7f251044fcc0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:41.416 7f251044fcc0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:41.416 7f251044fcc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:41.416 7f251044fcc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:41.417 7f251044fcc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:41.417 7f251044fcc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:41.417 7f251044fcc0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:41.417 7f251044fcc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:41.417 7f251044fcc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:41.417 7f251044fcc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:41.417 7f251044fcc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:41.417 7f251044fcc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:41.418 7f251044fcc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:27:41.418 7f251044fcc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:45.418 7f2510475b70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:50.421 7f2510475b70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:50.421 7f2510475b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:50.421 7f2510475b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:50.421 7f2510475b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:50.421 7f2510475b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:50.422 7f2510475b70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:50.422 7f2510475b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:50.422 7f2510475b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:50.422 7f2510475b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:50.422 7f2510475b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:50.422 7f2510475b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:50.422 7f2510475b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:27:50.423 7f2510475b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:50.506 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51410932, partition_space_available_1k=29679552, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102908, cpu_share=0.0, mem_res=97100, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84908, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93976, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189428, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42312, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15288508, free=9498128, buffers=996164, cached=6882320, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.84, five_min_avg=3.77, fifteen_min_avg=3.97, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:27:50.507 7f250cb38a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:50.507 7f250cb38a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51410932 partition_space_available_1k = 29679552 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102908 cpu_share = 0.0 mem_res = 97100 >>contrail-schema : << mem_virt = 84908 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 93976 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189428 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42312 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15288508 free = 9498128 buffers = 996164 cached = 6882320 node_type = config-node >> system_cpu_usage = << one_min_avg = 3.84 five_min_avg = 3.77 fifteen_min_avg = 3.97 cpu_share = 2.62 node_type = config-node >> >> 06/02/2025 02:27:50.507 7f250cb38a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:54.425 7f250cb38a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:27:59.430 7f250cb38a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:27:59.430 7f250cb38a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:27:59.430 7f250cb38a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:59.430 7f250cb38a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:27:59.430 7f250cb38a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:27:59.431 7f250cb38a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:27:59.431 7f250cb38a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:27:59.431 7f250cb38a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:59.431 7f250cb38a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:27:59.431 7f250cb38a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:59.431 7f250cb38a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:27:59.431 7f250cb38a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:27:59.431 7f250cb38a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:03.434 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:08.441 7f251170d630 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:08.441 7f251170d630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:08.441 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:08.441 7f251170d630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:08.442 7f251170d630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:08.442 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:08.442 7f251170d630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:08.442 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:08.442 7f251170d630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:08.442 7f251170d630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:08.442 7f251170d630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:08.443 7f251170d630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:28:08.443 7f251170d630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:12.442 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:17.449 7f250d57a4e0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:17.449 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:17.449 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:17.450 7f250d57a4e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:17.450 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:17.450 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:17.450 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:17.450 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:17.450 7f250d57a4e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:17.451 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:17.451 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:17.451 7f250d57a4e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:28:17.451 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:21.454 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:26.456 7f250d57a4e0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:26.456 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:26.456 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:26.456 7f250d57a4e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:26.456 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:26.457 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:26.457 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:26.457 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:26.457 7f250d57a4e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:26.457 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:26.457 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:26.457 7f250d57a4e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:28:26.458 7f250d57a4e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:30.459 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:35.465 7f250cb22780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:35.465 7f250cb22780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:35.465 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:35.466 7f250cb22780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:35.466 7f250cb22780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:35.466 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:35.466 7f250cb22780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:35.466 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:35.466 7f250cb22780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:35.466 7f250cb22780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:35.467 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:35.467 7f250cb22780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:28:35.467 7f250cb22780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:39.470 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:44.472 7f250cb22780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:44.473 7f250cb22780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:44.473 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:44.473 7f250cb22780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:44.473 7f250cb22780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:44.473 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:44.474 7f250cb22780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:44.474 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:44.474 7f250cb22780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:44.474 7f250cb22780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:44.474 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:44.474 7f250cb22780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:28:44.474 7f250cb22780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:48.475 7f250e212780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:28:50.509 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51416680, partition_space_available_1k=29673804, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102984, cpu_share=0.0, mem_res=97100, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84964, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94032, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189436, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42372, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15298488, free=9481824, buffers=996164, cached=6888644, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.35, five_min_avg=3.35, fifteen_min_avg=3.81, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:28:53.480 7f250e212780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:28:53.481 7f250e212780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:28:53.481 7f250e212780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:53.481 7f250e212780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:28:53.481 7f250e212780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:53.481 7f250e212780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:53.481 7f250e212780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51416680 partition_space_available_1k = 29673804 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102984 cpu_share = 0.0 mem_res = 97100 >>contrail-schema : << mem_virt = 84964 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94032 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189436 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42372 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15298488 free = 9481824 buffers = 996164 cached = 6888644 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.35 five_min_avg = 3.35 fifteen_min_avg = 3.81 cpu_share = 2.62 node_type = config-node >> >> 06/02/2025 02:28:53.481 7f250e212780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:28:53.481 7f250e212780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:28:53.482 7f250e212780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:28:53.482 7f250e212780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:53.482 7f250e212780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:28:53.482 7f250e212780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:53.482 7f250e212780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:53.482 7f250e212780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:28:53.482 7f250e212780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:28:57.481 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:02.487 7f250e226828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:02.488 7f250e226828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:02.488 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:02.488 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:02.488 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:02.489 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:02.489 7f250e226828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:02.489 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:02.489 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:02.490 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:02.490 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:02.490 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:29:02.490 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:06.492 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:11.496 7f250e226828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:11.497 7f250e226828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:11.497 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:11.497 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:11.497 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:11.497 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:11.497 7f250e226828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:11.498 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:11.498 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:11.498 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:11.498 7f250e226828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:11.498 7f250e226828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:29:11.498 7f250e226828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:15.501 7f251047a240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:20.506 7f251047a240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:20.506 7f251047a240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:20.506 7f251047a240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:20.506 7f251047a240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:20.506 7f251047a240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:20.507 7f251047a240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:20.507 7f251047a240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:20.507 7f251047a240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:20.507 7f251047a240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:20.507 7f251047a240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:20.508 7f251047a240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:20.508 7f251047a240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:29:20.508 7f251047a240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:24.510 7f251047a240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:29.514 7f251047a240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:29.514 7f251047a240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:29.514 7f251047a240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:29.514 7f251047a240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:29.515 7f251047a240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:29.515 7f251047a240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:29.515 7f251047a240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:29.516 7f251047a240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:29.516 7f251047a240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:29.516 7f251047a240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:29.516 7f251047a240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:29.516 7f251047a240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:29:29.516 7f251047a240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:33.518 7f250d57a048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:38.521 7f250d57a048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:38.521 7f250d57a048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:38.521 7f250d57a048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:38.521 7f250d57a048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:38.522 7f250d57a048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:38.522 7f250d57a048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:38.522 7f250d57a048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:38.522 7f250d57a048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:38.522 7f250d57a048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:38.522 7f250d57a048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:38.522 7f250d57a048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:38.522 7f250d57a048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:29:38.522 7f250d57a048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:42.522 7f251047ccc0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:47.525 7f251047ccc0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:47.526 7f251047ccc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:47.526 7f251047ccc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:47.526 7f251047ccc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:47.526 7f251047ccc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:47.526 7f251047ccc0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:47.527 7f251047ccc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:47.527 7f251047ccc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:47.527 7f251047ccc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:47.527 7f251047ccc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:47.527 7f251047ccc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:47.528 7f251047ccc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:29:47.528 7f251047ccc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:50.514 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51417164, partition_space_available_1k=29673320, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103052, cpu_share=0.0, mem_res=97160, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84988, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94064, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189504, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42356, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15284296, free=9494464, buffers=996180, cached=6890180, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.19, five_min_avg=3.14, fifteen_min_avg=3.71, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:29:50.514 7f250d57ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:50.514 7f250d57ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51417164 partition_space_available_1k = 29673320 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103052 cpu_share = 0.0 mem_res = 97160 >>contrail-schema : << mem_virt = 84988 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94064 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189504 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42356 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15284296 free = 9494464 buffers = 996180 cached = 6890180 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.19 five_min_avg = 3.14 fifteen_min_avg = 3.71 cpu_share = 2.5 node_type = config-node >> >> 06/02/2025 02:29:50.514 7f250d57ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:51.526 7f250d57ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:29:56.531 7f250d57ac18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:29:56.531 7f250d57ac18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:29:56.531 7f250d57ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:56.532 7f250d57ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:29:56.532 7f250d57ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:29:56.532 7f250d57ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:29:56.533 7f250d57ac18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:29:56.533 7f250d57ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:56.533 7f250d57ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:29:56.534 7f250d57ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:56.534 7f250d57ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:29:56.534 7f250d57ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:29:56.534 7f250d57ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:00.535 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:05.541 7f25164f4048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:05.542 7f25164f4048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:05.542 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:05.542 7f25164f4048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:05.542 7f25164f4048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:05.542 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:05.542 7f25164f4048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:05.543 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:05.543 7f25164f4048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:05.543 7f25164f4048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:05.543 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:05.543 7f25164f4048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:30:05.543 7f25164f4048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:09.545 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:14.549 7f25164f4048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:14.549 7f25164f4048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:14.549 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:14.549 7f25164f4048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:14.549 7f25164f4048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:14.549 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:14.550 7f25164f4048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:14.551 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:14.551 7f25164f4048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:14.551 7f25164f4048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:14.551 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:14.551 7f25164f4048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:30:14.551 7f25164f4048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:18.553 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:23.559 7f25164f4048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:23.559 7f25164f4048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:23.559 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:23.559 7f25164f4048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:23.560 7f25164f4048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:23.560 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:23.560 7f25164f4048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:23.560 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:23.560 7f25164f4048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:23.560 7f25164f4048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:23.560 7f25164f4048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:23.560 7f25164f4048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:30:23.560 7f25164f4048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:27.560 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:32.566 7f250cb38e10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:32.566 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:32.566 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:32.566 7f250cb38e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:32.566 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:32.566 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:32.567 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:32.567 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:32.567 7f250cb38e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:32.567 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:32.567 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:32.567 7f250cb38e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:30:32.567 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:36.571 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:41.574 7f250cb38e10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:41.574 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:41.575 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:41.575 7f250cb38e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:41.575 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:41.575 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:41.576 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:41.576 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:41.576 7f250cb38e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:41.576 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:41.576 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:41.576 7f250cb38e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:30:41.576 7f250cb38e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:45.576 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:50.526 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51422384, partition_space_available_1k=29668100, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103188, cpu_share=0.0, mem_res=97260, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84988, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94064, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189508, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42412, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15353456, free=9387528, buffers=997764, cached=6926372, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.1, five_min_avg=3.97, fifteen_min_avg=3.95, cpu_share=5.0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:30:50.577 7f250dd73eb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:50.578 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:50.578 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:50.578 7f250dd73eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:50.578 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:50.578 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:50.578 7f250dd73eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51422384 partition_space_available_1k = 29668100 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103188 cpu_share = 0.0 mem_res = 97260 >>contrail-schema : << mem_virt = 84988 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94064 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189508 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42412 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15353456 free = 9387528 buffers = 997764 cached = 6926372 node_type = config-node >> system_cpu_usage = << one_min_avg = 6.1 five_min_avg = 3.97 fifteen_min_avg = 3.95 cpu_share = 5.0 node_type = config-node >> >> 06/02/2025 02:30:50.578 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:50.579 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:50.579 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:50.579 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:50.579 7f250dd73eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:50.579 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:50.579 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:50.580 7f250dd73eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:30:50.580 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:52.471 7f250dd73eb8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:30:54.580 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:30:59.583 7f250dd73eb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:30:59.583 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:30:59.583 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:59.583 7f250dd73eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:30:59.583 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:30:59.583 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:30:59.584 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:30:59.584 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:59.584 7f250dd73eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:30:59.584 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:59.584 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:30:59.584 7f250dd73eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:30:59.584 7f250dd73eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:02.873 7f251009f588 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:31:03.584 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:08.590 7f251009f588 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:08.591 7f251009f588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:08.591 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:08.591 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:08.591 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:08.591 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:08.592 7f251009f588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:08.592 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:08.592 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:08.592 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:08.592 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:08.592 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:31:08.592 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:12.592 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:13.438 7f2510606a20 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:31:17.597 7f251009f588 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:17.597 7f251009f588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:17.597 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:17.597 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:17.598 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:17.598 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:17.598 7f251009f588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:17.598 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:17.598 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:17.598 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:17.598 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:17.598 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:31:17.598 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:21.601 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:26.607 7f251009f588 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:26.608 7f251009f588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:26.608 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:26.608 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:26.608 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:26.608 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:26.609 7f251009f588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:26.609 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:26.609 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:26.609 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:26.609 7f251009f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:26.609 7f251009f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:31:26.609 7f251009f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:30.610 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:35.616 7f251047c8d0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:35.617 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:35.617 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:35.617 7f251047c8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:35.617 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:35.617 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:35.617 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:35.617 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:35.618 7f251047c8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:35.618 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:35.618 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:35.618 7f251047c8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:31:35.618 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:39.620 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:44.622 7f251047c8d0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:44.623 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:44.623 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:44.623 7f251047c8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:44.623 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:44.623 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:44.624 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:44.624 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:44.624 7f251047c8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:44.624 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:44.624 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:44.625 7f251047c8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:31:44.625 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:48.623 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:31:50.526 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51428656, partition_space_available_1k=29661828, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103260, cpu_share=0.0, mem_res=97260, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85040, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94116, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189548, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42444, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15312128, free=9423260, buffers=997792, cached=6931940, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.52, five_min_avg=3.65, fifteen_min_avg=3.84, cpu_share=2.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:31:53.627 7f251047c8d0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:31:53.628 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:31:53.628 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:53.628 7f251047c8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:31:53.628 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:53.628 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:53.628 7f251047c8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51428656 partition_space_available_1k = 29661828 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103260 cpu_share = 0.0 mem_res = 97260 >>contrail-schema : << mem_virt = 85040 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94116 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189548 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42444 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15312128 free = 9423260 buffers = 997792 cached = 6931940 node_type = config-node >> system_cpu_usage = << one_min_avg = 3.52 five_min_avg = 3.65 fifteen_min_avg = 3.84 cpu_share = 2.88 node_type = config-node >> >> 06/02/2025 02:31:53.628 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:31:53.628 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:31:53.629 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:31:53.629 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:53.629 7f251047c8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:31:53.629 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:53.629 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:53.629 7f251047c8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:31:53.630 7f251047c8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:31:57.631 7f2510488b70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:02.637 7f2510488b70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:02.637 7f2510488b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:02.637 7f2510488b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:02.637 7f2510488b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:02.637 7f2510488b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:02.637 7f2510488b70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:02.638 7f2510488b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:02.638 7f2510488b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:02.638 7f2510488b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:02.638 7f2510488b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:02.638 7f2510488b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:02.638 7f2510488b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:32:02.638 7f2510488b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:06.641 7f251743e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:11.647 7f251743e0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:11.648 7f251743e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:11.648 7f251743e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:11.648 7f251743e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:11.648 7f251743e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:11.648 7f251743e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:11.648 7f251743e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:11.649 7f251743e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:11.649 7f251743e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:11.649 7f251743e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:11.649 7f251743e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:11.649 7f251743e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:32:11.649 7f251743e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:15.650 7f250d580978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:20.656 7f250d580978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:20.656 7f250d580978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:20.656 7f250d580978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:20.656 7f250d580978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:20.656 7f250d580978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:20.657 7f250d580978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:20.657 7f250d580978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:20.657 7f250d580978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:20.657 7f250d580978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:20.657 7f250d580978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:20.657 7f250d580978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:20.657 7f250d580978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:32:20.657 7f250d580978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:24.657 7f250d580978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:29.659 7f250d580978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:29.659 7f250d580978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:29.659 7f250d580978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:29.660 7f250d580978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:29.660 7f250d580978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:29.660 7f250d580978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:29.660 7f250d580978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:29.660 7f250d580978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:29.661 7f250d580978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:29.661 7f250d580978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:29.661 7f250d580978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:29.661 7f250d580978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:32:29.661 7f250d580978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:30.415 7f250e2126d8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:32:33.663 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:38.669 7f250e2126d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:38.670 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:38.670 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:38.670 7f250e2126d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:38.670 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:38.670 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:38.671 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:38.671 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:38.671 7f250e2126d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:38.671 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:38.671 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:38.671 7f250e2126d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:32:38.671 7f250e2126d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:42.670 7f250c9c6a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:47.674 7f250c9c6a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:47.674 7f250c9c6a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:47.674 7f250c9c6a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:47.674 7f250c9c6a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:47.674 7f250c9c6a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:47.675 7f250c9c6a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:47.675 7f250c9c6a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:47.675 7f250c9c6a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:47.675 7f250c9c6a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:47.675 7f250c9c6a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:47.675 7f250c9c6a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:47.675 7f250c9c6a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:32:47.675 7f250c9c6a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:50.523 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51430604, partition_space_available_1k=29659880, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103268, cpu_share=0.0, mem_res=97260, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85036, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94148, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189616, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42448, cpu_share=0.01, mem_res=39640, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15335692, free=9397000, buffers=997792, cached=6934636, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.93, five_min_avg=3.42, fifteen_min_avg=3.75, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:32:50.523 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:50.524 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51430604 partition_space_available_1k = 29659880 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103268 cpu_share = 0.0 mem_res = 97260 >>contrail-schema : << mem_virt = 85036 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94148 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189616 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42448 cpu_share = 0.01 mem_res = 39640 >> } system_mem_usage = << total = 32665120 used = 15335692 free = 9397000 buffers = 997792 cached = 6934636 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.93 five_min_avg = 3.42 fifteen_min_avg = 3.75 cpu_share = 2.62 node_type = config-node >> >> 06/02/2025 02:32:50.524 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:51.675 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:32:56.681 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:32:56.682 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:32:56.682 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:56.682 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:32:56.682 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:32:56.682 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:32:56.682 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:32:56.683 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:56.683 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:32:56.683 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:56.683 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:32:56.683 7f2511ae9d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:32:56.683 7f2511ae9d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:00.683 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:05.688 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:05.688 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:05.688 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:05.688 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:05.688 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:05.688 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:05.689 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:05.689 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:05.689 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:05.689 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:05.689 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:05.689 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:33:05.689 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:09.691 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:14.693 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:14.693 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:14.693 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:14.694 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:14.694 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:14.694 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:14.694 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:14.694 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:14.694 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:14.694 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:14.695 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:14.695 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:33:14.695 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:18.697 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:23.703 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:23.703 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:23.703 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:23.703 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:23.703 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:23.703 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:23.704 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:23.704 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:23.704 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:23.704 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:23.704 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:23.704 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:33:23.704 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:27.707 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:32.713 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:32.713 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:32.713 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:32.713 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:32.713 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:32.713 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:32.714 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:32.714 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:32.714 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:32.714 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:32.714 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:32.714 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:33:32.714 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:36.716 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:41.719 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:41.719 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:41.720 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:41.720 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:41.720 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:41.720 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:41.721 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:41.721 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:41.721 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:41.721 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:41.721 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:41.721 7f250cb38b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:33:41.721 7f250cb38b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:45.721 7f250dd8f828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:50.527 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51436376, partition_space_available_1k=29654108, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103300, cpu_share=0.0, mem_res=97260, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85060, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94140, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189616, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42512, cpu_share=0.01, mem_res=39648, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15304316, free=9421684, buffers=997792, cached=6941328, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.32, five_min_avg=3.13, fifteen_min_avg=3.62, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:33:50.723 7f250dd8f828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:50.724 7f250dd8f828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:50.724 7f250dd8f828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:50.724 7f250dd8f828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:50.724 7f250dd8f828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:50.724 7f250dd8f828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:50.724 7f250dd8f828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51436376 partition_space_available_1k = 29654108 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103300 cpu_share = 0.0 mem_res = 97260 >>contrail-schema : << mem_virt = 85060 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94140 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189616 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42512 cpu_share = 0.01 mem_res = 39648 >> } system_mem_usage = << total = 32665120 used = 15304316 free = 9421684 buffers = 997792 cached = 6941328 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.32 five_min_avg = 3.13 fifteen_min_avg = 3.62 cpu_share = 2.62 node_type = config-node >> >> 06/02/2025 02:33:50.724 7f250dd8f828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:50.724 7f250dd8f828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:50.725 7f250dd8f828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:50.725 7f250dd8f828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:50.725 7f250dd8f828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:50.725 7f250dd8f828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:50.725 7f250dd8f828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:50.725 7f250dd8f828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:33:50.725 7f250dd8f828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:54.728 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:33:59.731 7f250d592a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:33:59.731 7f250d592a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:33:59.731 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:59.731 7f250d592a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:33:59.731 7f250d592a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:33:59.731 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:33:59.731 7f250d592a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:33:59.732 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:59.732 7f250d592a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:33:59.732 7f250d592a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:59.732 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:33:59.732 7f250d592a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:33:59.732 7f250d592a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:03.734 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:08.740 7f250d592a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:08.740 7f250d592a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:08.740 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:08.740 7f250d592a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:08.740 7f250d592a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:08.740 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:08.741 7f250d592a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:08.741 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:08.741 7f250d592a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:08.741 7f250d592a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:08.741 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:08.742 7f250d592a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:34:08.742 7f250d592a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:12.744 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:17.749 7f250d592a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:17.749 7f250d592a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:17.749 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:17.749 7f250d592a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:17.749 7f250d592a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:17.749 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:17.750 7f250d592a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:17.750 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:17.750 7f250d592a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:17.750 7f250d592a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:17.750 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:17.750 7f250d592a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:34:17.750 7f250d592a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:21.753 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:26.759 7f250d592a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:26.759 7f250d592a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:26.759 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:26.760 7f250d592a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:26.760 7f250d592a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:26.760 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:26.760 7f250d592a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:26.760 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:26.761 7f250d592a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:26.761 7f250d592a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:26.761 7f250d592a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:26.761 7f250d592a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:34:26.761 7f250d592a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:30.760 7f25104790f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:35.766 7f25104790f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:35.766 7f25104790f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:35.766 7f25104790f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:35.767 7f25104790f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:35.767 7f25104790f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:35.767 7f25104790f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:35.767 7f25104790f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:35.767 7f25104790f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:35.767 7f25104790f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:35.767 7f25104790f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:35.767 7f25104790f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:35.767 7f25104790f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:34:35.768 7f25104790f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:39.771 7f25104790f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:44.773 7f25104790f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:44.773 7f25104790f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:44.773 7f25104790f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:44.773 7f25104790f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:44.773 7f25104790f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:44.774 7f25104790f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:44.774 7f25104790f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:44.774 7f25104790f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:44.774 7f25104790f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:44.774 7f25104790f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:44.774 7f25104790f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:44.774 7f25104790f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:34:44.775 7f25104790f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:48.774 7f2510488ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:34:50.528 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51437048, partition_space_available_1k=29653436, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103372, cpu_share=0.0, mem_res=97260, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85116, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94196, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189656, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42532, cpu_share=0.01, mem_res=39648, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15308820, free=9415828, buffers=997912, cached=6942560, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.78, five_min_avg=2.83, fifteen_min_avg=3.49, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:34:53.779 7f2510488ac8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:34:53.779 7f2510488ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:34:53.779 7f2510488ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:53.779 7f2510488ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:34:53.779 7f2510488ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:53.780 7f2510488ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:53.780 7f2510488ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51437048 partition_space_available_1k = 29653436 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103372 cpu_share = 0.0 mem_res = 97260 >>contrail-schema : << mem_virt = 85116 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94196 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189656 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42532 cpu_share = 0.01 mem_res = 39648 >> } system_mem_usage = << total = 32665120 used = 15308820 free = 9415828 buffers = 997912 cached = 6942560 node_type = config-node >> system_cpu_usage = << one_min_avg = 1.78 five_min_avg = 2.83 fifteen_min_avg = 3.49 cpu_share = 2.75 node_type = config-node >> >> 06/02/2025 02:34:53.780 7f2510488ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:34:53.780 7f2510488ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:34:53.781 7f2510488ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:34:53.781 7f2510488ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:53.781 7f2510488ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:34:53.781 7f2510488ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:53.781 7f2510488ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:53.781 7f2510488ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:34:53.781 7f2510488ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:34:57.781 7f2510606048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:02.785 7f2510606048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:02.785 7f2510606048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:02.785 7f2510606048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:02.786 7f2510606048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:02.786 7f2510606048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:02.786 7f2510606048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:02.786 7f2510606048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:02.786 7f2510606048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:02.786 7f2510606048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:02.787 7f2510606048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:02.787 7f2510606048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:02.787 7f2510606048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:35:02.787 7f2510606048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:06.789 7f2510606048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:11.795 7f2510606048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:11.795 7f2510606048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:11.795 7f2510606048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:11.795 7f2510606048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:11.796 7f2510606048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:11.796 7f2510606048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:11.796 7f2510606048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:11.796 7f2510606048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:11.796 7f2510606048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:11.797 7f2510606048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:11.797 7f2510606048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:11.797 7f2510606048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:35:11.797 7f2510606048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:15.796 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:20.803 7f250d57aeb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:20.803 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:20.803 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:20.803 7f250d57aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:20.803 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:20.804 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:20.804 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:20.804 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:20.806 7f250d57aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:20.806 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:20.806 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:20.806 7f250d57aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:35:20.806 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:24.806 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:29.809 7f250d57aeb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:29.810 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:29.810 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:29.810 7f250d57aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:29.810 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:29.810 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:29.811 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:29.811 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:29.811 7f250d57aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:29.811 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:29.811 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:29.811 7f250d57aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:35:29.811 7f250d57aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:33.813 7f250dd8f588 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:38.820 7f250dd8f588 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:38.820 7f250dd8f588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:38.820 7f250dd8f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:38.820 7f250dd8f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:38.821 7f250dd8f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:38.821 7f250dd8f588 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:38.822 7f250dd8f588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:38.822 7f250dd8f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:38.822 7f250dd8f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:38.822 7f250dd8f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:38.822 7f250dd8f588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:38.822 7f250dd8f588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:35:38.823 7f250dd8f588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:42.820 7f250c9c6438 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:47.824 7f250c9c6438 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:47.824 7f250c9c6438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:47.824 7f250c9c6438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:47.825 7f250c9c6438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:47.825 7f250c9c6438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:47.825 7f250c9c6438 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:47.825 7f250c9c6438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:47.826 7f250c9c6438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:47.826 7f250c9c6438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:47.826 7f250c9c6438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:47.826 7f250c9c6438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:47.826 7f250c9c6438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:35:47.826 7f250c9c6438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:50.536 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51437704, partition_space_available_1k=29652780, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103380, cpu_share=0.0, mem_res=97260, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85108, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94224, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189720, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42524, cpu_share=0.01, mem_res=39648, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15317340, free=9406040, buffers=997912, cached=6943828, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.98, five_min_avg=2.7, fifteen_min_avg=3.4, cpu_share=2.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:35:50.536 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:50.537 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51437704 partition_space_available_1k = 29652780 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103380 cpu_share = 0.0 mem_res = 97260 >>contrail-schema : << mem_virt = 85108 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94224 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189720 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42524 cpu_share = 0.01 mem_res = 39648 >> } system_mem_usage = << total = 32665120 used = 15317340 free = 9406040 buffers = 997912 cached = 6943828 node_type = config-node >> system_cpu_usage = << one_min_avg = 1.98 five_min_avg = 2.7 fifteen_min_avg = 3.4 cpu_share = 2.88 node_type = config-node >> >> 06/02/2025 02:35:50.537 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:51.825 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:35:56.831 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:35:56.832 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:35:56.832 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:56.832 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:35:56.832 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:35:56.832 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:35:56.833 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:35:56.833 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:56.833 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:35:56.833 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:56.833 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:35:56.834 7f25100f9a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:35:56.834 7f25100f9a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:00.832 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:05.838 7f2511b3d780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:05.838 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:05.838 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:05.838 7f2511b3d780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:05.838 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:05.838 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:05.839 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:05.839 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:05.839 7f2511b3d780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:05.839 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:05.839 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:05.839 7f2511b3d780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:36:05.840 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:08.750 7f2511b3d780 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:36:09.838 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:14.845 7f2511b3d780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:14.891 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:14.891 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:14.891 7f2511b3d780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:14.892 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:14.892 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:14.892 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:14.892 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:14.892 7f2511b3d780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:14.892 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:14.892 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:14.892 7f2511b3d780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:36:14.893 7f2511b3d780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:18.848 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:19.178 7f2511707ac8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:36:23.853 7f250dd8fa20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:23.853 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:23.853 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:23.853 7f250dd8fa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:23.853 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:23.853 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:23.854 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:23.854 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:23.854 7f250dd8fa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:23.854 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:23.854 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:23.855 7f250dd8fa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:36:23.855 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:27.857 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:32.864 7f250dd8fa20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:32.865 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:32.865 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:32.865 7f250dd8fa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:32.865 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:32.865 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:32.866 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:32.866 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:32.866 7f250dd8fa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:32.866 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:32.866 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:32.867 7f250dd8fa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:36:32.867 7f250dd8fa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:34.697 7f250cb1ee10 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:36:36.866 7f250cb1ee10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:41.869 7f250cb1ee10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:41.869 7f250cb1ee10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:41.869 7f250cb1ee10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:41.869 7f250cb1ee10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:41.869 7f250cb1ee10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:41.869 7f250cb1ee10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:41.870 7f250cb1ee10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:41.870 7f250cb1ee10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:41.870 7f250cb1ee10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:41.870 7f250cb1ee10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:41.870 7f250cb1ee10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:41.870 7f250cb1ee10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:36:41.870 7f250cb1ee10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:45.870 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:50.541 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51447916, partition_space_available_1k=29642568, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103420, cpu_share=0.0, mem_res=97260, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85140, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94216, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189724, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42580, cpu_share=0.01, mem_res=39648, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15380528, free=9321748, buffers=997988, cached=6964856, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.66, five_min_avg=3.54, fifteen_min_avg=3.64, cpu_share=5.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:36:50.873 7f250d5920f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:50.873 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:50.873 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:50.873 7f250d5920f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:50.873 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:50.873 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:50.873 7f250d5920f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51447916 partition_space_available_1k = 29642568 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103420 cpu_share = 0.0 mem_res = 97260 >>contrail-schema : << mem_virt = 85140 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94216 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189724 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42580 cpu_share = 0.01 mem_res = 39648 >> } system_mem_usage = << total = 32665120 used = 15380528 free = 9321748 buffers = 997988 cached = 6964856 node_type = config-node >> system_cpu_usage = << one_min_avg = 5.66 five_min_avg = 3.54 fifteen_min_avg = 3.64 cpu_share = 5.38 node_type = config-node >> >> 06/02/2025 02:36:50.873 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:50.873 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:50.874 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:50.874 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:50.874 7f250d5920f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:50.874 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:50.875 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:50.875 7f250d5920f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:36:50.875 7f250d5920f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:54.876 7f2511766f60 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:36:59.882 7f2511766f60 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:36:59.882 7f2511766f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:36:59.882 7f2511766f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:59.882 7f2511766f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:36:59.883 7f2511766f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:36:59.883 7f2511766f60 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:36:59.883 7f2511766f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:36:59.883 7f2511766f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:59.883 7f2511766f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:36:59.883 7f2511766f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:59.883 7f2511766f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:36:59.883 7f2511766f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:36:59.883 7f2511766f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:03.885 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:08.891 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:08.891 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:08.891 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:08.892 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:08.892 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:08.892 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:08.893 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:08.893 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:08.893 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:08.893 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:08.893 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:08.893 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:37:08.893 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:12.895 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:17.898 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:17.898 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:17.898 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:17.898 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:17.898 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:17.898 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:17.898 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:17.899 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:17.899 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:17.899 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:17.899 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:17.899 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:37:17.899 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:21.900 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:26.905 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:26.905 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:26.905 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:26.905 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:26.905 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:26.905 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:26.906 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:26.906 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:26.906 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:26.906 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:26.906 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:26.906 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:37:26.906 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:30.909 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:35.914 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:35.914 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:35.914 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:35.915 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:35.915 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:35.915 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:35.915 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:35.915 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:35.915 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:35.915 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:35.915 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:35.916 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:37:35.916 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:39.918 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:44.920 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:44.920 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:44.921 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:44.921 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:44.921 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:44.921 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:44.922 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:44.922 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:44.922 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:44.922 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:44.922 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:44.922 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:37:44.923 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:48.921 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:37:50.537 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51455752, partition_space_available_1k=29634732, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103484, cpu_share=0.0, mem_res=97260, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85192, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94276, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189764, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42580, cpu_share=0.01, mem_res=39620, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15365340, free=9333788, buffers=997988, cached=6968004, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.05, five_min_avg=3.47, fifteen_min_avg=3.61, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:37:53.925 7f250d57d828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:37:53.925 7f250d57d828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:37:53.925 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:53.925 7f250d57d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:37:53.925 7f250d57d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:53.925 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:53.925 7f250d57d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51455752 partition_space_available_1k = 29634732 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103484 cpu_share = 0.0 mem_res = 97260 >>contrail-schema : << mem_virt = 85192 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94276 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189764 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42580 cpu_share = 0.01 mem_res = 39620 >> } system_mem_usage = << total = 32665120 used = 15365340 free = 9333788 buffers = 997988 cached = 6968004 node_type = config-node >> system_cpu_usage = << one_min_avg = 4.05 five_min_avg = 3.47 fifteen_min_avg = 3.61 cpu_share = 2.62 node_type = config-node >> >> 06/02/2025 02:37:53.926 7f250d57d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:37:53.926 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:37:53.926 7f250d57d828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:37:53.926 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:53.926 7f250d57d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:37:53.926 7f250d57d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:53.926 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:53.926 7f250d57d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:37:53.926 7f250d57d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:37:57.928 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:02.932 7f250d57d828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:02.932 7f250d57d828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:02.932 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:02.932 7f250d57d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:02.932 7f250d57d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:02.932 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:02.933 7f250d57d828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:02.933 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:02.933 7f250d57d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:02.933 7f250d57d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:02.933 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:02.934 7f250d57d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:38:02.934 7f250d57d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:06.933 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:11.937 7f250d57d828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:11.937 7f250d57d828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:11.937 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:11.937 7f250d57d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:11.937 7f250d57d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:11.937 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:11.938 7f250d57d828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:11.938 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:11.938 7f250d57d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:11.938 7f250d57d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:11.938 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:11.938 7f250d57d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:38:11.938 7f250d57d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:15.941 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:18.794 7f251047ee10 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:38:20.943 7f250d57d828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:20.944 7f250d57d828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:20.944 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:20.944 7f250d57d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:20.944 7f250d57d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:20.944 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:20.944 7f250d57d828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:20.945 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:20.945 7f250d57d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:20.945 7f250d57d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:20.945 7f250d57d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:20.945 7f250d57d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:38:20.945 7f250d57d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:24.948 7f25104600f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:29.958 7f25104600f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:29.958 7f25104600f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:29.958 7f25104600f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:29.958 7f25104600f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:29.958 7f25104600f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:29.958 7f25104600f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:29.959 7f25104600f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:29.959 7f25104600f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:29.959 7f25104600f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:29.959 7f25104600f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:29.959 7f25104600f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:29.959 7f25104600f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:38:29.959 7f25104600f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:33.959 7f251048ae10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:38.965 7f251048ae10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:38.965 7f251048ae10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:38.966 7f251048ae10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:38.966 7f251048ae10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:38.966 7f251048ae10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:38.966 7f251048ae10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:38.966 7f251048ae10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:38.966 7f251048ae10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:38.967 7f251048ae10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:38.967 7f251048ae10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:38.967 7f251048ae10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:38.967 7f251048ae10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:38:38.967 7f251048ae10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:42.966 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:47.968 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:47.968 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:47.968 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:47.969 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:47.969 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:47.969 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:47.969 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:47.969 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:47.970 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:47.970 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:47.970 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:47.970 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:38:47.970 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:50.543 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51456820, partition_space_available_1k=29633664, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103528, cpu_share=0.0, mem_res=97292, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85188, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94304, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189828, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42596, cpu_share=0.01, mem_res=39648, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15351424, free=9340900, buffers=997996, cached=6974800, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.04, five_min_avg=3.3, fifteen_min_avg=3.54, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:38:50.544 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:50.544 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51456820 partition_space_available_1k = 29633664 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103528 cpu_share = 0.0 mem_res = 97292 >>contrail-schema : << mem_virt = 85188 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94304 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189828 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42596 cpu_share = 0.01 mem_res = 39648 >> } system_mem_usage = << total = 32665120 used = 15351424 free = 9340900 buffers = 997996 cached = 6974800 node_type = config-node >> system_cpu_usage = << one_min_avg = 3.04 five_min_avg = 3.3 fifteen_min_avg = 3.54 cpu_share = 2.62 node_type = config-node >> >> 06/02/2025 02:38:50.544 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:51.970 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:38:56.976 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:38:56.977 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:38:56.977 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:56.977 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:38:56.977 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:38:56.977 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:38:56.977 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:38:56.977 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:56.978 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:38:56.978 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:56.978 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:38:56.978 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:38:56.978 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:00.981 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:05.985 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:05.986 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:05.986 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:05.986 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:05.986 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:05.986 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:05.986 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:05.986 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:05.987 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:05.987 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:05.987 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:05.987 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:39:05.987 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:09.989 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:14.992 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:14.992 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:14.992 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:14.993 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:14.993 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:14.993 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:14.993 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:14.993 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:14.993 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:14.994 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:14.994 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:14.994 7f25157f3f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:39:14.994 7f25157f3f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:18.992 7f250d57d240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:23.996 7f250d57d240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:23.997 7f250d57d240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:23.997 7f250d57d240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:23.997 7f250d57d240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:23.997 7f250d57d240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:23.997 7f250d57d240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:23.998 7f250d57d240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:23.998 7f250d57d240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:23.998 7f250d57d240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:23.998 7f250d57d240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:23.998 7f250d57d240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:23.998 7f250d57d240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:39:23.998 7f250d57d240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:28.001 7f250d57d240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:33.003 7f250d57d240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:33.004 7f250d57d240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:33.004 7f250d57d240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:33.004 7f250d57d240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:33.004 7f250d57d240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:33.004 7f250d57d240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:33.005 7f250d57d240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:33.005 7f250d57d240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:33.005 7f250d57d240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:33.005 7f250d57d240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:33.005 7f250d57d240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:33.005 7f250d57d240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:39:33.005 7f250d57d240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:37.008 7f250dd732e8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:42.011 7f250dd732e8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:42.011 7f250dd732e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:42.011 7f250dd732e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:42.012 7f250dd732e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:42.012 7f250dd732e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:42.012 7f250dd732e8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:42.013 7f250dd732e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:42.013 7f250dd732e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:42.013 7f250dd732e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:42.014 7f250dd732e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:42.014 7f250dd732e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:42.014 7f250dd732e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:39:42.014 7f250dd732e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:46.013 7f251047eeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:39:50.544 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51457504, partition_space_available_1k=29632980, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103564, cpu_share=0.0, mem_res=97292, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85216, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94300, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189840, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42672, cpu_share=0.01, mem_res=39668, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15356296, free=9331880, buffers=997996, cached=6978948, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.3, five_min_avg=3.04, fifteen_min_avg=3.44, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:39:51.015 7f251047eeb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:39:51.015 7f251047eeb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:39:51.015 7f251047eeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:51.015 7f251047eeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:39:51.015 7f251047eeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:51.016 7f251047eeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:51.016 7f251047eeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51457504 partition_space_available_1k = 29632980 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103564 cpu_share = 0.0 mem_res = 97292 >>contrail-schema : << mem_virt = 85216 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94300 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189840 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42672 cpu_share = 0.01 mem_res = 39668 >> } system_mem_usage = << total = 32665120 used = 15356296 free = 9331880 buffers = 997996 cached = 6978948 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.3 five_min_avg = 3.04 fifteen_min_avg = 3.44 cpu_share = 2.5 node_type = config-node >> >> 06/02/2025 02:39:51.016 7f251047eeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:39:51.016 7f251047eeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:39:51.016 7f251047eeb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:39:51.016 7f251047eeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:51.017 7f251047eeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:39:51.017 7f251047eeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:51.017 7f251047eeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:51.017 7f251047eeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:39:51.017 7f251047eeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:39:55.018 7f250cb266d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:00.024 7f250cb266d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:00.025 7f250cb266d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:00.025 7f250cb266d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:00.025 7f250cb266d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:00.025 7f250cb266d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:00.025 7f250cb266d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:00.026 7f250cb266d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:00.026 7f250cb266d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:00.026 7f250cb266d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:00.026 7f250cb266d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:00.026 7f250cb266d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:00.026 7f250cb266d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:40:00.027 7f250cb266d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:04.025 7f250d5904e0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:09.032 7f250d5904e0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:09.032 7f250d5904e0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:09.032 7f250d5904e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:09.033 7f250d5904e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:09.033 7f250d5904e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:09.033 7f250d5904e0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:09.034 7f250d5904e0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:09.034 7f250d5904e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:09.034 7f250d5904e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:09.034 7f250d5904e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:09.034 7f250d5904e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:09.034 7f250d5904e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:40:09.034 7f250d5904e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:12.086 7f250d5904e0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:40:13.033 7f2510479390 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:18.038 7f2510479390 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:18.039 7f2510479390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:18.039 7f2510479390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:18.039 7f2510479390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:18.039 7f2510479390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:18.039 7f2510479390 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:18.039 7f2510479390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:18.040 7f2510479390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:18.040 7f2510479390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:18.040 7f2510479390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:18.040 7f2510479390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:18.040 7f2510479390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:40:18.040 7f2510479390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:22.040 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:22.498 7f250d56f630 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:40:27.046 7f251066a390 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:27.047 7f251066a390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:27.047 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:27.047 7f251066a390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:27.047 7f251066a390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:27.047 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:27.048 7f251066a390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:27.049 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:27.049 7f251066a390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:27.049 7f251066a390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:27.049 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:27.049 7f251066a390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:40:27.049 7f251066a390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:31.050 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:34.418 7f250cb1ee10 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:40:36.054 7f251066a390 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:36.054 7f251066a390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:36.054 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:36.054 7f251066a390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:36.054 7f251066a390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:36.054 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:36.055 7f251066a390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:36.055 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:36.055 7f251066a390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:36.055 7f251066a390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:36.055 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:36.056 7f251066a390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:40:36.056 7f251066a390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:40.058 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:45.061 7f251066a390 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:45.061 7f251066a390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:45.061 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:45.061 7f251066a390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:45.061 7f251066a390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:45.061 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:45.062 7f251066a390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:45.062 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:45.062 7f251066a390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:45.062 7f251066a390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:45.062 7f251066a390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:45.063 7f251066a390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:40:45.063 7f251066a390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:49.062 7f2510488240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:40:50.557 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51465096, partition_space_available_1k=29625388, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103640, cpu_share=0.0, mem_res=97292, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85268, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94360, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189872, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42692, cpu_share=0.01, mem_res=39668, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15499960, free=9168296, buffers=998008, cached=6998856, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.82, five_min_avg=4.53, fifteen_min_avg=3.91, cpu_share=5.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:40:54.065 7f2510488240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:40:54.065 7f2510488240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:40:54.065 7f2510488240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:54.065 7f2510488240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:40:54.065 7f2510488240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:54.065 7f2510488240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:54.065 7f2510488240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51465096 partition_space_available_1k = 29625388 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103640 cpu_share = 0.0 mem_res = 97292 >>contrail-schema : << mem_virt = 85268 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94360 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189872 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42692 cpu_share = 0.01 mem_res = 39668 >> } system_mem_usage = << total = 32665120 used = 15499960 free = 9168296 buffers = 998008 cached = 6998856 node_type = config-node >> system_cpu_usage = << one_min_avg = 8.82 five_min_avg = 4.53 fifteen_min_avg = 3.91 cpu_share = 5.25 node_type = config-node >> >> 06/02/2025 02:40:54.065 7f2510488240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:40:54.065 7f2510488240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:40:54.066 7f2510488240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:40:54.066 7f2510488240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:54.066 7f2510488240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:40:54.066 7f2510488240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:54.066 7f2510488240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:54.066 7f2510488240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:40:54.066 7f2510488240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:40:58.069 7f250c9c6ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:03.077 7f250c9c6ac8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:03.077 7f250c9c6ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:03.077 7f250c9c6ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:03.077 7f250c9c6ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:03.077 7f250c9c6ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:03.077 7f250c9c6ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:03.078 7f250c9c6ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:03.078 7f250c9c6ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:03.078 7f250c9c6ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:03.078 7f250c9c6ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:03.078 7f250c9c6ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:03.078 7f250c9c6ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:41:03.078 7f250c9c6ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:07.078 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:12.083 7f2516896a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:12.083 7f2516896a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:12.083 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:12.083 7f2516896a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:12.083 7f2516896a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:12.083 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:12.084 7f2516896a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:12.084 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:12.084 7f2516896a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:12.084 7f2516896a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:12.084 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:12.084 7f2516896a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:41:12.084 7f2516896a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:16.087 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:21.090 7f2516896a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:21.091 7f2516896a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:21.091 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:21.091 7f2516896a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:21.091 7f2516896a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:21.091 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:21.091 7f2516896a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:21.091 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:21.092 7f2516896a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:21.092 7f2516896a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:21.092 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:21.092 7f2516896a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:41:21.092 7f2516896a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:25.095 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:30.101 7f2516896a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:30.102 7f2516896a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:30.102 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:30.102 7f2516896a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:30.102 7f2516896a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:30.102 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:30.103 7f2516896a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:30.103 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:30.103 7f2516896a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:30.103 7f2516896a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:30.103 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:30.103 7f2516896a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:41:30.103 7f2516896a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:34.105 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:39.111 7f2516896a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:39.112 7f2516896a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:39.112 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:39.112 7f2516896a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:39.112 7f2516896a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:39.112 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:39.112 7f2516896a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:39.113 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:39.113 7f2516896a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:39.113 7f2516896a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:39.113 7f2516896a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:39.113 7f2516896a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:41:39.113 7f2516896a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:43.113 7f251044f780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:48.117 7f251044f780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:48.117 7f251044f780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:48.117 7f251044f780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:48.117 7f251044f780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:48.117 7f251044f780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:48.118 7f251044f780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:48.118 7f251044f780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:48.118 7f251044f780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:48.118 7f251044f780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:48.118 7f251044f780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:48.118 7f251044f780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:48.118 7f251044f780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:41:48.118 7f251044f780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:50.560 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51470124, partition_space_available_1k=29620360, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103644, cpu_share=0.0, mem_res=97292, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85264, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94388, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189936, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42692, cpu_share=0.01, mem_res=39668, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15418428, free=9246016, buffers=998008, cached=7002668, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.29, five_min_avg=4.03, fifteen_min_avg=3.78, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:41:50.561 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:50.561 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51470124 partition_space_available_1k = 29620360 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103644 cpu_share = 0.0 mem_res = 97292 >>contrail-schema : << mem_virt = 85264 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94388 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189936 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42692 cpu_share = 0.01 mem_res = 39668 >> } system_mem_usage = << total = 32665120 used = 15418428 free = 9246016 buffers = 998008 cached = 7002668 node_type = config-node >> system_cpu_usage = << one_min_avg = 4.29 five_min_avg = 4.03 fifteen_min_avg = 3.78 cpu_share = 2.75 node_type = config-node >> >> 06/02/2025 02:41:50.561 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:52.119 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:41:57.125 7f25100f8978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:41:57.125 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:41:57.125 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:57.125 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:41:57.125 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:41:57.125 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:41:57.126 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:41:57.126 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:57.126 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:41:57.126 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:57.126 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:41:57.126 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:41:57.126 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:01.129 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:06.131 7f25100f8978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:06.131 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:06.131 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:06.131 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:06.132 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:06.132 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:06.132 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:06.132 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:06.132 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:06.132 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:06.132 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:06.133 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:42:06.133 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:10.134 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:15.139 7f25100f8978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:15.140 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:15.140 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:15.140 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:15.140 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:15.140 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:15.140 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:15.140 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:15.141 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:15.141 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:15.141 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:15.141 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:42:15.141 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:19.144 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:24.150 7f25100f8978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:24.150 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:24.150 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:24.150 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:24.150 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:24.150 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:24.151 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:24.151 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:24.151 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:24.151 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:24.151 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:24.151 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:42:24.151 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:28.154 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:33.161 7f25100f8978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:33.161 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:33.161 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:33.161 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:33.161 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:33.161 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:33.162 7f25100f8978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:33.162 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:33.162 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:33.162 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:33.162 7f25100f8978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:33.162 7f25100f8978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:42:33.162 7f25100f8978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:37.163 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:42.165 7f2510606a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:42.166 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:42.166 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:42.167 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:42.167 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:42.167 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:42.167 7f2510606a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:42.167 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:42.168 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:42.168 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:42.168 7f2510606a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:42.168 7f2510606a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:42:42.168 7f2510606a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:46.166 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:42:50.560 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51476256, partition_space_available_1k=29614228, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103684, cpu_share=0.0, mem_res=97292, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85292, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94388, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189940, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42756, cpu_share=0.01, mem_res=39676, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15417932, free=9239208, buffers=998008, cached=7009972, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.48, five_min_avg=3.86, fifteen_min_avg=3.73, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:42:51.169 7f250d592b70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:42:51.169 7f250d592b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:42:51.169 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:51.169 7f250d592b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:42:51.169 7f250d592b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:51.169 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:51.169 7f250d592b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51476256 partition_space_available_1k = 29614228 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103684 cpu_share = 0.0 mem_res = 97292 >>contrail-schema : << mem_virt = 85292 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94388 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189940 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42756 cpu_share = 0.01 mem_res = 39676 >> } system_mem_usage = << total = 32665120 used = 15417932 free = 9239208 buffers = 998008 cached = 7009972 node_type = config-node >> system_cpu_usage = << one_min_avg = 3.48 five_min_avg = 3.86 fifteen_min_avg = 3.73 cpu_share = 2.62 node_type = config-node >> >> 06/02/2025 02:42:51.169 7f250d592b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:42:51.169 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:42:51.170 7f250d592b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:42:51.170 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:51.170 7f250d592b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:42:51.170 7f250d592b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:51.170 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:51.170 7f250d592b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:42:51.171 7f250d592b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:42:55.172 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:00.178 7f250d592b70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:00.178 7f250d592b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:00.179 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:00.179 7f250d592b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:00.179 7f250d592b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:00.179 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:00.179 7f250d592b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:00.179 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:00.180 7f250d592b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:00.180 7f250d592b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:00.180 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:00.180 7f250d592b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:43:00.180 7f250d592b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:04.182 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:09.187 7f250d592b70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:09.187 7f250d592b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:09.187 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:09.188 7f250d592b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:09.188 7f250d592b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:09.188 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:09.188 7f250d592b70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:09.188 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:09.189 7f250d592b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:09.189 7f250d592b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:09.189 7f250d592b70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:09.189 7f250d592b70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:43:09.189 7f250d592b70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:10.790 7f250d57deb8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:43:13.190 7f250d57deb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:18.197 7f250d57deb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:18.197 7f250d57deb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:18.197 7f250d57deb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:18.197 7f250d57deb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:18.197 7f250d57deb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:18.197 7f250d57deb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:18.198 7f250d57deb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:18.198 7f250d57deb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:18.198 7f250d57deb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:18.198 7f250d57deb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:18.198 7f250d57deb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:18.198 7f250d57deb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:43:18.198 7f250d57deb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:22.199 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:27.205 7f251047a0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:27.206 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:27.206 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:27.206 7f251047a0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:27.206 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:27.206 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:27.207 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:27.207 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:27.207 7f251047a0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:27.207 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:27.207 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:27.208 7f251047a0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:43:27.208 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:31.210 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:36.213 7f251047a0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:36.213 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:36.213 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:36.214 7f251047a0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:36.214 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:36.214 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:36.215 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:36.215 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:36.215 7f251047a0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:36.215 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:36.215 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:36.215 7f251047a0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:43:36.215 7f251047a0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:40.217 7f250d56fb70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:45.220 7f250d56fb70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:45.221 7f250d56fb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:45.221 7f250d56fb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:45.221 7f250d56fb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:45.221 7f250d56fb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:45.221 7f250d56fb70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:45.222 7f250d56fb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:45.222 7f250d56fb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:45.222 7f250d56fb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:45.223 7f250d56fb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:45.223 7f250d56fb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:45.223 7f250d56fb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:43:45.223 7f250d56fb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:49.221 7f2510460048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:43:50.548 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51477644, partition_space_available_1k=29612840, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103748, cpu_share=0.0, mem_res=97292, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85348, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94440, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189980, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42776, cpu_share=0.01, mem_res=39676, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15407208, free=9247308, buffers=998008, cached=7012596, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.04, five_min_avg=3.68, fifteen_min_avg=3.68, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:43:54.225 7f2510460048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:43:54.226 7f2510460048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:43:54.226 7f2510460048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:54.226 7f2510460048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:43:54.226 7f2510460048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:54.226 7f2510460048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:54.226 7f2510460048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51477644 partition_space_available_1k = 29612840 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103748 cpu_share = 0.0 mem_res = 97292 >>contrail-schema : << mem_virt = 85348 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94440 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 189980 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42776 cpu_share = 0.01 mem_res = 39676 >> } system_mem_usage = << total = 32665120 used = 15407208 free = 9247308 buffers = 998008 cached = 7012596 node_type = config-node >> system_cpu_usage = << one_min_avg = 3.04 five_min_avg = 3.68 fifteen_min_avg = 3.68 cpu_share = 2.75 node_type = config-node >> >> 06/02/2025 02:43:54.226 7f2510460048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:43:54.226 7f2510460048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:43:54.227 7f2510460048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:43:54.227 7f2510460048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:54.228 7f2510460048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:43:54.228 7f2510460048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:54.228 7f2510460048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:54.228 7f2510460048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:43:54.228 7f2510460048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:43:58.229 7f250d57a588 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:03.233 7f250d57a588 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:03.233 7f250d57a588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:03.233 7f250d57a588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:03.233 7f250d57a588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:03.233 7f250d57a588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:03.233 7f250d57a588 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:03.234 7f250d57a588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:03.234 7f250d57a588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:03.234 7f250d57a588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:03.234 7f250d57a588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:03.234 7f250d57a588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:03.234 7f250d57a588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:44:03.235 7f250d57a588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:07.234 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:12.239 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:12.239 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:12.239 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:12.239 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:12.239 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:12.239 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:12.240 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:12.240 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:12.240 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:12.240 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:12.240 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:12.240 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:44:12.240 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:16.243 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:21.244 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:21.245 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:21.245 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:21.245 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:21.245 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:21.245 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:21.245 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:21.245 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:21.245 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:21.246 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:21.246 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:21.246 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:44:21.246 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:25.249 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:30.255 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:30.256 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:30.256 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:30.256 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:30.256 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:30.256 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:30.256 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:30.256 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:30.257 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:30.257 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:30.257 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:30.257 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:44:30.257 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:34.260 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:39.265 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:39.265 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:39.265 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:39.265 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:39.265 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:39.265 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:39.266 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:39.266 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:39.266 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:39.266 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:39.266 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:39.266 7f250d56fe10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:44:39.266 7f250d56fe10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:43.266 7f250d5926d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:48.270 7f250d5926d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:48.270 7f250d5926d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:48.270 7f250d5926d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:48.270 7f250d5926d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:48.270 7f250d5926d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:48.270 7f250d5926d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:48.271 7f250d5926d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:48.271 7f250d5926d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:48.271 7f250d5926d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:48.271 7f250d5926d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:48.271 7f250d5926d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:48.271 7f250d5926d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:44:48.271 7f250d5926d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:50.559 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51478304, partition_space_available_1k=29612180, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103760, cpu_share=0.0, mem_res=97292, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85340, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94468, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=190044, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42764, cpu_share=0.01, mem_res=39676, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15408640, free=9244412, buffers=998008, cached=7014060, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.51, five_min_avg=3.39, fifteen_min_avg=3.58, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:44:50.560 7f250cb26978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:50.560 7f250cb26978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51478304 partition_space_available_1k = 29612180 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103760 cpu_share = 0.0 mem_res = 97292 >>contrail-schema : << mem_virt = 85340 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94468 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 190044 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42764 cpu_share = 0.01 mem_res = 39676 >> } system_mem_usage = << total = 32665120 used = 15408640 free = 9244412 buffers = 998008 cached = 7014060 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.51 five_min_avg = 3.39 fifteen_min_avg = 3.58 cpu_share = 2.75 node_type = config-node >> >> 06/02/2025 02:44:50.560 7f250cb26978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:52.271 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:44:57.277 7f250cb22780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:44:57.277 7f250cb22780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:44:57.277 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:57.277 7f250cb22780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:44:57.277 7f250cb22780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:44:57.277 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:44:57.278 7f250cb22780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:44:57.278 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:57.278 7f250cb22780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:44:57.278 7f250cb22780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:57.278 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:44:57.278 7f250cb22780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:44:57.278 7f250cb22780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:01.281 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:06.289 7f250cb22780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:06.289 7f250cb22780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:06.289 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:06.289 7f250cb22780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:06.289 7f250cb22780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:06.289 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:06.290 7f250cb22780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:06.290 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:06.290 7f250cb22780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:06.290 7f250cb22780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:06.290 7f250cb22780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:06.290 7f250cb22780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:45:06.290 7f250cb22780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:10.291 7f2507f15828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:15.297 7f2507f15828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:15.297 7f2507f15828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:15.297 7f2507f15828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:15.297 7f2507f15828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:15.297 7f2507f15828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:15.297 7f2507f15828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:15.298 7f2507f15828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:15.298 7f2507f15828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:15.298 7f2507f15828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:15.298 7f2507f15828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:15.298 7f2507f15828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:15.298 7f2507f15828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:45:15.298 7f2507f15828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:19.301 7f2507f15828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:24.307 7f2507f15828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:24.307 7f2507f15828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:24.307 7f2507f15828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:24.308 7f2507f15828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:24.308 7f2507f15828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:24.308 7f2507f15828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:24.308 7f2507f15828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:24.308 7f2507f15828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:24.308 7f2507f15828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:24.308 7f2507f15828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:24.308 7f2507f15828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:24.308 7f2507f15828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:45:24.309 7f2507f15828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:28.311 7f251047ec18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:33.317 7f251047ec18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:33.317 7f251047ec18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:33.317 7f251047ec18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:33.317 7f251047ec18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:33.317 7f251047ec18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:33.317 7f251047ec18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:33.318 7f251047ec18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:33.318 7f251047ec18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:33.318 7f251047ec18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:33.318 7f251047ec18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:33.318 7f251047ec18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:33.318 7f251047ec18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:45:33.318 7f251047ec18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:37.319 7f250dd8f2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:42.323 7f250dd8f2e8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:42.323 7f250dd8f2e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:42.324 7f250dd8f2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:42.324 7f250dd8f2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:42.324 7f250dd8f2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:42.326 7f250dd8f2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:42.326 7f250dd8f2e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:42.327 7f250dd8f2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:42.327 7f250dd8f2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:42.327 7f250dd8f2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:42.327 7f250dd8f2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:42.327 7f250dd8f2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:45:42.327 7f250dd8f2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:46.325 7f251046e2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:49.903 7f2507f20240 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:45:50.560 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51484924, partition_space_available_1k=29605560, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103800, cpu_share=0.0, mem_res=97292, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85368, cpu_share=0.0, mem_res=78796, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94464, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=190052, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42816, cpu_share=0.01, mem_res=39676, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15520840, free=9110248, buffers=998008, cached=7036024, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=7.08, five_min_avg=4.57, fifteen_min_avg=3.97, cpu_share=5.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:45:51.328 7f251046e2e8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:45:51.328 7f251046e2e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:45:51.328 7f251046e2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:51.328 7f251046e2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:45:51.328 7f251046e2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:51.328 7f251046e2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:51.329 7f251046e2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51484924 partition_space_available_1k = 29605560 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103800 cpu_share = 0.0 mem_res = 97292 >>contrail-schema : << mem_virt = 85368 cpu_share = 0.0 mem_res = 78796 >>contrail-svc-monitor : << mem_virt = 94464 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 190052 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42816 cpu_share = 0.01 mem_res = 39676 >> } system_mem_usage = << total = 32665120 used = 15520840 free = 9110248 buffers = 998008 cached = 7036024 node_type = config-node >> system_cpu_usage = << one_min_avg = 7.08 five_min_avg = 4.57 fifteen_min_avg = 3.97 cpu_share = 5.25 node_type = config-node >> >> 06/02/2025 02:45:51.329 7f251046e2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:45:51.329 7f251046e2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:45:51.329 7f251046e2e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:45:51.329 7f251046e2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:51.329 7f251046e2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:45:51.329 7f251046e2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:51.329 7f251046e2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:51.330 7f251046e2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:45:51.330 7f251046e2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:45:55.329 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:45:59.989 7f25164f4048 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:46:00.331 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:00.332 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:00.332 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:00.333 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:00.333 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:00.333 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:00.333 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:00.333 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:00.333 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:00.333 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:00.334 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:00.334 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:46:00.334 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:04.333 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:09.336 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:09.337 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:09.337 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:09.337 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:09.337 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:09.337 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:09.338 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:09.338 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:09.339 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:09.339 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:09.339 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:09.339 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:46:09.339 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:10.903 7f250cb22cc0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:46:13.338 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:18.345 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:18.345 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:18.345 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:18.345 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:18.345 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:18.346 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:18.346 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:18.346 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:18.346 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:18.347 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:18.347 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:18.347 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:46:18.347 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:22.349 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:27.353 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:27.353 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:27.353 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:27.353 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:27.353 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:27.354 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:27.354 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:27.354 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:27.355 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:27.355 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:27.355 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:27.355 7f250cb22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:46:27.355 7f250cb22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:31.357 7f251047a8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:36.363 7f251047a8d0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:36.363 7f251047a8d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:36.363 7f251047a8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:36.363 7f251047a8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:36.364 7f251047a8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:36.364 7f251047a8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:36.364 7f251047a8d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:36.364 7f251047a8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:36.364 7f251047a8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:36.364 7f251047a8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:36.364 7f251047a8d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:36.364 7f251047a8d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:46:36.364 7f251047a8d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:40.365 7f251046e4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:45.367 7f251046e4e0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:45.368 7f251046e4e0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:45.368 7f251046e4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:45.368 7f251046e4e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:45.368 7f251046e4e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:45.368 7f251046e4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:45.369 7f251046e4e0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:45.369 7f251046e4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:45.369 7f251046e4e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:45.369 7f251046e4e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:45.369 7f251046e4e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:45.369 7f251046e4e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:46:45.370 7f251046e4e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:49.368 7f25104600f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:46:50.574 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51490624, partition_space_available_1k=29599860, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103872, cpu_share=0.0, mem_res=97292, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85088, cpu_share=0.0, mem_res=78540, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94520, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=190092, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42844, cpu_share=0.01, mem_res=39676, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15432780, free=9196392, buffers=998012, cached=7037936, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.95, five_min_avg=4.14, fifteen_min_avg=3.86, cpu_share=3.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:46:54.372 7f25104600f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:46:54.372 7f25104600f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:46:54.372 7f25104600f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:54.373 7f25104600f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:46:54.373 7f25104600f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:54.373 7f25104600f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:54.373 7f25104600f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51490624 partition_space_available_1k = 29599860 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103872 cpu_share = 0.0 mem_res = 97292 >>contrail-schema : << mem_virt = 85088 cpu_share = 0.0 mem_res = 78540 >>contrail-svc-monitor : << mem_virt = 94520 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 190092 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42844 cpu_share = 0.01 mem_res = 39676 >> } system_mem_usage = << total = 32665120 used = 15432780 free = 9196392 buffers = 998012 cached = 7037936 node_type = config-node >> system_cpu_usage = << one_min_avg = 3.95 five_min_avg = 4.14 fifteen_min_avg = 3.86 cpu_share = 3.12 node_type = config-node >> >> 06/02/2025 02:46:54.373 7f25104600f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:46:54.373 7f25104600f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:46:54.374 7f25104600f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:46:54.374 7f25104600f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:54.374 7f25104600f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:46:54.374 7f25104600f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:54.374 7f25104600f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:54.374 7f25104600f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:46:54.374 7f25104600f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:46:58.374 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:03.381 7f250d57f0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:03.381 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:03.381 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:03.382 7f250d57f0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:03.382 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:03.382 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:03.382 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:03.382 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:03.382 7f250d57f0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:03.383 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:03.383 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:03.383 7f250d57f0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:47:03.383 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:07.385 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:12.391 7f250d57f0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:12.392 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:12.392 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:12.392 7f250d57f0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:12.393 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:12.393 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:12.393 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:12.393 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:12.394 7f250d57f0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:12.394 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:12.394 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:12.394 7f250d57f0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:47:12.394 7f250d57f0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:16.395 7f250cb34a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:21.368 7f251047a8d0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:47:21.398 7f250cb34a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:21.398 7f250cb34a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:21.399 7f250cb34a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:21.399 7f250cb34a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:21.400 7f250cb34a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:21.400 7f250cb34a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:21.402 7f250cb34a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:21.402 7f250cb34a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:21.402 7f250cb34a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:21.403 7f250cb34a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:21.403 7f250cb34a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:21.403 7f250cb34a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:47:21.403 7f250cb34a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:25.399 7f250d580e10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:30.406 7f250d580e10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:30.406 7f250d580e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:30.406 7f250d580e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:30.406 7f250d580e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:30.406 7f250d580e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:30.406 7f250d580e10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:30.407 7f250d580e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:30.407 7f250d580e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:30.407 7f250d580e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:30.407 7f250d580e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:30.407 7f250d580e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:30.407 7f250d580e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:47:30.408 7f250d580e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:34.410 7f250d580e10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:39.413 7f250d580e10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:39.413 7f250d580e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:39.413 7f250d580e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:39.413 7f250d580e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:39.414 7f250d580e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:39.414 7f250d580e10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:39.414 7f250d580e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:39.414 7f250d580e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:39.414 7f250d580e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:39.414 7f250d580e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:39.414 7f250d580e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:39.415 7f250d580e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:47:39.415 7f250d580e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:43.415 7f25104602e8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:48.419 7f25104602e8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:48.419 7f25104602e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:48.420 7f25104602e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:48.420 7f25104602e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:48.420 7f25104602e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:48.420 7f25104602e8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:48.421 7f25104602e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:48.421 7f25104602e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:48.421 7f25104602e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:48.421 7f25104602e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:48.421 7f25104602e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:48.422 7f25104602e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:47:48.422 7f25104602e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:50.577 7f251065d1c8 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-542-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51497176, partition_space_available_1k=29593308, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103948, cpu_share=0.0, mem_res=97360, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85080, cpu_share=0.0, mem_res=78540, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=94544, cpu_share=0.0, mem_res=85100, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=190156, cpu_share=0.0, mem_res=176012, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42832, cpu_share=0.01, mem_res=39676, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=15422244, free=9199732, buffers=998012, cached=7045132, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.8, five_min_avg=4.06, fifteen_min_avg=3.85, cpu_share=3.0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/02/2025 02:47:50.578 7f250d580390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:50.578 7f250d580390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51497176 partition_space_available_1k = 29593308 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 103948 cpu_share = 0.0 mem_res = 97360 >>contrail-schema : << mem_virt = 85080 cpu_share = 0.0 mem_res = 78540 >>contrail-svc-monitor : << mem_virt = 94544 cpu_share = 0.0 mem_res = 85100 >>contrail-device-manager : << mem_virt = 190156 cpu_share = 0.0 mem_res = 176012 >>contrail-config-nodemgr : << mem_virt = 42832 cpu_share = 0.01 mem_res = 39676 >> } system_mem_usage = << total = 32665120 used = 15422244 free = 9199732 buffers = 998012 cached = 7045132 node_type = config-node >> system_cpu_usage = << one_min_avg = 3.8 five_min_avg = 4.06 fifteen_min_avg = 3.85 cpu_share = 3.0 node_type = config-node >> >> 06/02/2025 02:47:50.578 7f250d580390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:52.421 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:47:57.426 7f251048a2e8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:47:57.426 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:47:57.426 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:57.426 7f251048a2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:47:57.427 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:47:57.427 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:47:57.427 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:47:57.427 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:57.427 7f251048a2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:47:57.427 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:57.427 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:47:57.428 7f251048a2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:47:57.428 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:01.429 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:48:06.433 7f251048a2e8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:48:06.433 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:48:06.433 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:06.433 7f251048a2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:48:06.433 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:06.433 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:48:06.434 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:48:06.434 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:06.434 7f251048a2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:48:06.434 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:06.434 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:06.434 7f251048a2e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:48:06.434 7f251048a2e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:10.434 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:48:15.439 7f250dd73e10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:48:15.439 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:48:15.439 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:15.439 7f250dd73e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:48:15.440 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:15.440 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:48:15.440 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:48:15.440 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:15.440 7f250dd73e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:48:15.440 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:15.440 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:15.440 7f250dd73e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:48:15.440 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:19.444 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/02/2025 02:48:23.201 7f250c9c64e0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/02/2025 02:48:24.447 7f250dd73e10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/02/2025 02:48:24.447 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/02/2025 02:48:24.447 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:24.447 7f250dd73e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/02/2025 02:48:24.448 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/02/2025 02:48:24.448 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/02/2025 02:48:24.448 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/02/2025 02:48:24.448 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:24.448 7f250dd73e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-542-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.129:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/02/2025 02:48:24.449 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:24.449 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:24.449 7f250dd73e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-542-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1748830000294000 collector_name = collector_ip = 10.0.0.129:8086 collector_list = [ 10.0.0.129:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/02/2025 02:48:24.449 7f250dd73e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/02/2025 02:48:28.449 7f2510488a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]