06/09/2025 01:50:46.458 7f6af16de908 [contrail-config-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 06/09/2025 01:50:46.458 7f6af16de908 [contrail-config-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 06/09/2025 01:50:46.458 7f6af16de908 [contrail-config-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 06/09/2025 01:50:46.473 7f6aef04b048 [contrail-config-nodemgr] [ERROR]: Cannot write http_port 8100 to /tmp/contrail-config-nodemgr.0.http_port 06/09/2025 01:50:46.475 7f6aef04b048 [contrail-config-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8100 06/09/2025 01:50:46.476 7f6aef050978 [contrail-config-nodemgr] [INFO]: Processing event[EvStart] in state[none] 06/09/2025 01:50:46.482 7f6aef050978 [contrail-config-nodemgr] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:50:46.483 7f6aef050978 [contrail-config-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/09/2025 01:50:46.487 7f6aef050978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 06/09/2025 01:50:46.487 7f6aef050978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:46.487 7f6aef050978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-544-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/09/2025 01:50:46.487 7f6aef050978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:46.487 7f6aef050978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:46.487 7f6aef050978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-544-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1749433846483548 collector_name = collector_ip = collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 06/09/2025 01:50:46.487 7f6aef050978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:46.585 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'contrail-api', 'group': 'contrail-api', 'pid': 68825336577276976611160514382080268425963532429460710728697208263525821187689, 'start': '1749433843000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-schema', 'group': 'contrail-schema', 'pid': 92747092018029143191871291418744696392109966894821286502263336881054349915847, 'start': '1749433843000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-svc-monitor', 'group': 'contrail-svc-monitor', 'pid': 18539078251551383786476998542640317578648069597856777344720353989684617868096, 'start': '1749433843000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-device-manager', 'group': 'contrail-device-manager', 'pid': 34666704612863748864334197059394332403712244036352409561413948882456926243894, 'start': '1749433843000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-nodemgr', 'group': 'contrail-config-nodemgr', 'pid': 57157308041186646529722449474137374049462323303954460673329277148503966614913, 'start': '1749433843000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 06/09/2025 01:50:47.032 7f6af16de908 [contrail-config-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-544-1', _node_type='Config', _instance_id='0', _timestamp=1749433847032460, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 06/09/2025 01:50:47.033 7f6af16de908 [contrail-config-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-544-1', _node_type='Config', _instance_id='0', _timestamp=1749433847033487, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-api', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1749433843000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-schema', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1749433843000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-svc-monitor', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1749433843000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-device-manager', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1749433843000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1749433843000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-09 00:44:55.879161", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9411-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 06/09/2025 01:50:47.038 7f6aeee71588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:47.039 7f6aeee71588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-544-1 build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-09 00:44:55.879161", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9411-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 06/09/2025 01:50:47.039 7f6aeee71588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:47.039 7f6aeee71588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:47.039 7f6aeee71588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-544-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/09/2025 01:50:47.039 7f6aeee71588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:47.039 7f6aeee71588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:47.040 7f6aeee71588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-544-1 deleted = False process_info = [ << process_name = contrail-api process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1749433843000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-schema process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1749433843000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-svc-monitor process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1749433843000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-device-manager process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1749433843000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1749433843000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-09 00:44:55.879161", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9411-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 06/09/2025 01:50:47.040 7f6aeee71588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:50.483 7f6aee789240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/09/2025 01:50:50.484 7f6aee789240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/09/2025 01:50:50.484 7f6aee789240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/09/2025 01:50:50.484 7f6aee789240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/09/2025 01:50:50.484 7f6aee789240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-544-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/09/2025 01:50:50.484 7f6aee789240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/09/2025 01:50:50.484 7f6aee789240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/09/2025 01:50:50.485 7f6aee789240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/09/2025 01:50:50.485 7f6aee789240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:50.485 7f6aee789240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-544-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/09/2025 01:50:50.485 7f6aee789240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:50.485 7f6aee789240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:50.485 7f6aee789240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-544-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1749433846483548 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/09/2025 01:50:50.485 7f6aee789240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:52.279 7f6aee789b70 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:50:54.485 7f6aee789cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/09/2025 01:50:54.486 7f6aee789cc0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/09/2025 01:50:54.486 7f6aee789cc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/09/2025 01:50:54.486 7f6aee789cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/09/2025 01:50:54.486 7f6aee789cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-544-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/09/2025 01:50:54.486 7f6aee789cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/09/2025 01:50:54.487 7f6aee789cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/09/2025 01:50:54.487 7f6aee789cc0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/09/2025 01:50:54.487 7f6aee789cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:54.488 7f6aee789cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-544-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/09/2025 01:50:54.488 7f6aee789cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:54.488 7f6aee789cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:54.488 7f6aee789cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-544-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1749433846483548 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 06/09/2025 01:50:54.488 7f6aee789cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:56.119 7f6af16de908 [contrail-config-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-544-1', _node_type='Config', _instance_id='0', _timestamp=1749433856119894, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-api', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1749433843000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-schema', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1749433843000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-svc-monitor', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1749433843000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-device-manager', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1749433843000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1749433843000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-09 00:44:55.879161", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9411-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 06/09/2025 01:50:56.120 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50805260, partition_space_available_1k=30285224, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=3600, cpu_share=0.0, mem_res=804, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2788, cpu_share=0.0, mem_res=800, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=3172, cpu_share=0.0, mem_res=800, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1656, cpu_share=0.0, mem_res=804, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80704, cpu_share=0.0, mem_res=39632, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=11738568, free=454120, buffers=897236, cached=19575188, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.67, five_min_avg=4.13, fifteen_min_avg=2.77, cpu_share=0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 01:50:56.120 7f6aedc22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:56.120 7f6aedc22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-544-1 deleted = False process_info = [ << process_name = contrail-api process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1749433843000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-schema process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1749433843000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-svc-monitor process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1749433843000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-device-manager process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1749433843000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1749433843000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-06-09 00:44:55.879161", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9411-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 06/09/2025 01:50:56.120 7f6aedc22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:56.120 7f6aedc22cc0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:56.121 7f6aedc22cc0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-544-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50805260 partition_space_available_1k = 30285224 percentage_partition_space_used = 63 >> } all_core_file_list = [ ] process_mem_cpu_usage = { contrail-api : << mem_virt = 3600 cpu_share = 0.0 mem_res = 804 >>contrail-schema : << mem_virt = 2788 cpu_share = 0.0 mem_res = 800 >>contrail-svc-monitor : << mem_virt = 3172 cpu_share = 0.0 mem_res = 800 >>contrail-device-manager : << mem_virt = 1656 cpu_share = 0.0 mem_res = 804 >>contrail-config-nodemgr : << mem_virt = 80704 cpu_share = 0.0 mem_res = 39632 >> } system_mem_usage = << total = 32665112 used = 11738568 free = 454120 buffers = 897236 cached = 19575188 node_type = config-node >> system_cpu_usage = << one_min_avg = 5.67 five_min_avg = 4.13 fifteen_min_avg = 2.77 cpu_share = 0 node_type = config-node >> >> 06/09/2025 01:50:56.121 7f6aedc22cc0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/09/2025 01:50:58.489 7f6aeee81a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/09/2025 01:50:58.495 7f6aeee81a20 [contrail-config-nodemgr] [INFO]: Session Event: TCP Connected 06/09/2025 01:50:58.495 7f6aeee81a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/09/2025 01:50:58.495 7f6aeee81a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/09/2025 01:50:58.495 7f6aeee81a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-544-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/09/2025 01:50:58.495 7f6aeee81a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/09/2025 01:50:58.495 7f6aeee81a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 06/09/2025 01:50:58.496 7f6aeee81a20 [contrail-config-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 06/09/2025 01:50:58.496 7f6aeee81a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 06/09/2025 01:50:58.496 7f6aeee81a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 06/09/2025 01:50:58.496 7f6aeee81a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 06/09/2025 01:50:58.499 7f6aedc22cc0 [contrail-config-nodemgr] [DEBUG]: Extracted sandesh length: 0000001011 06/09/2025 01:50:58.500 7f6aedc22cc0 [contrail-config-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 06/09/2025 01:50:58.500 7f6aeee8fd68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 06/09/2025 01:50:58.500 7f6aeee8fd68 [contrail-config-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 06/09/2025 01:50:58.501 7f6aeee8fd68 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:50:58.501 7f6aeee8fd68 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/09/2025 01:50:58.501 7f6aeee8fd68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 06/09/2025 01:50:58.501 7f6af14f30f0 [contrail-config-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 06/09/2025 01:50:58.502 7f6af14f30f0 [contrail-config-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 06/09/2025 01:51:06.692 7f6aef04b828 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:51:08.769 7f6aedc22cc0 [contrail-config-nodemgr] [DEBUG]: Extracted sandesh length: 0000001290 06/09/2025 01:51:08.770 7f6aedc22cc0 [contrail-config-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 06/09/2025 01:51:08.770 7f6af14c7ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 06/09/2025 01:51:08.771 7f6af14c7ac8 [contrail-config-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 2 06/09/2025 01:51:08.771 7f6af14c7ac8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:51:08.771 7f6af14c7ac8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/09/2025 01:51:16.464 7f6af14c7ac8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:51:27.959 7f6aef050cc0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:51:44.352 7f6aef018630 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:51:53.404 7f6aee78b0f0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:51:56.107 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51432564, partition_space_available_1k=29657920, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=98984, cpu_share=0.12, mem_res=93952, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80144, cpu_share=0.05, mem_res=77256, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88804, cpu_share=0.07, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=183308, cpu_share=0.09, mem_res=176268, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58212, cpu_share=0.01, mem_res=39624, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15003108, free=11911936, buffers=681556, cached=5068512, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=13.58, five_min_avg=6.52, fifteen_min_avg=3.67, cpu_share=8.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 01:51:57.512 7f6aef39da20 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:52:03.801 7f6aedc22e10 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:52:30.446 7f6aef0188d0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:52:56.098 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51469880, partition_space_available_1k=29620604, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102016, cpu_share=0.1, mem_res=96096, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80584, cpu_share=0.01, mem_res=77400, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=89316, cpu_share=0.01, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=183784, cpu_share=0.05, mem_res=176628, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58080, cpu_share=0.01, mem_res=39624, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15068444, free=11810512, buffers=689912, cached=5096244, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.7, five_min_avg=6.47, fifteen_min_avg=3.83, cpu_share=5.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 01:53:56.111 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51391292, partition_space_available_1k=29699192, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102032, cpu_share=0.0, mem_res=96096, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80584, cpu_share=0.0, mem_res=77400, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=89316, cpu_share=0.0, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=183796, cpu_share=0.0, mem_res=176628, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58088, cpu_share=0.01, mem_res=39624, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15107156, free=10267968, buffers=751072, cached=6538916, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.34, five_min_avg=6.95, fifteen_min_avg=4.17, cpu_share=4.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 01:54:56.132 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51400064, partition_space_available_1k=29690420, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102040, cpu_share=0.0, mem_res=96096, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80584, cpu_share=0.0, mem_res=77400, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=89320, cpu_share=0.0, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=183808, cpu_share=0.0, mem_res=176628, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58084, cpu_share=0.01, mem_res=39624, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15124868, free=10236936, buffers=751796, cached=6551512, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.74, five_min_avg=6.15, fifteen_min_avg=4.06, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 01:55:56.153 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51406852, partition_space_available_1k=29683632, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102052, cpu_share=0.0, mem_res=96096, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80584, cpu_share=0.0, mem_res=77400, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=89316, cpu_share=0.0, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=183816, cpu_share=0.0, mem_res=176628, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58420, cpu_share=0.01, mem_res=39624, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15149796, free=10200164, buffers=751816, cached=6563336, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.47, five_min_avg=5.5, fifteen_min_avg=3.97, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 01:56:16.840 7f6aee7a72e8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:56:21.819 7f6aee7934e0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:56:26.819 7f6aee7934e0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:56:56.153 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51418372, partition_space_available_1k=29672112, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102064, cpu_share=0.0, mem_res=96096, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80584, cpu_share=0.0, mem_res=77400, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=89316, cpu_share=0.0, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=183824, cpu_share=0.0, mem_res=176628, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58424, cpu_share=0.01, mem_res=39624, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15141776, free=10185024, buffers=752300, cached=6586012, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.51, five_min_avg=4.82, fifteen_min_avg=3.83, cpu_share=2.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 01:57:56.163 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51429824, partition_space_available_1k=29660660, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102152, cpu_share=0.0, mem_res=96168, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80584, cpu_share=0.0, mem_res=77400, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=89316, cpu_share=0.0, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=183836, cpu_share=0.0, mem_res=176628, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58428, cpu_share=0.01, mem_res=39624, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15092212, free=10208680, buffers=752312, cached=6611908, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.07, five_min_avg=5.01, fifteen_min_avg=3.95, cpu_share=4.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 01:58:12.739 7f6aeee81390 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 01:58:56.181 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51450360, partition_space_available_1k=29640124, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102364, cpu_share=0.0, mem_res=96356, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80584, cpu_share=0.0, mem_res=77400, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=89320, cpu_share=0.0, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=183848, cpu_share=0.0, mem_res=176632, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58428, cpu_share=0.01, mem_res=39624, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15274196, free=9608080, buffers=755444, cached=7027392, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.51, five_min_avg=4.55, fifteen_min_avg=3.86, cpu_share=4.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 01:59:56.180 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=52557344, partition_space_available_1k=28533140, percentage_partition_space_used=65, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102368, cpu_share=0.0, mem_res=96356, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80584, cpu_share=0.0, mem_res=77400, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=89320, cpu_share=0.0, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=183860, cpu_share=0.0, mem_res=176632, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58428, cpu_share=0.01, mem_res=39624, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15581404, free=8377056, buffers=782828, cached=7923824, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.82, five_min_avg=4.75, fifteen_min_avg=3.97, cpu_share=4.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 01:59:59.205 7f6af16fd390 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 02:00:56.203 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=52597900, partition_space_available_1k=28492584, percentage_partition_space_used=65, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=106652, cpu_share=0.05, mem_res=99740, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=81176, cpu_share=0.01, mem_res=77772, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=89580, cpu_share=0.01, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=184380, cpu_share=0.01, mem_res=176892, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58716, cpu_share=0.01, mem_res=39848, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=16767076, free=7107700, buffers=785992, cached=8004344, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.7, five_min_avg=4.91, fifteen_min_avg=4.07, cpu_share=4.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 02:01:56.201 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=52622080, partition_space_available_1k=28468404, percentage_partition_space_used=65, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=112876, cpu_share=0.1, mem_res=104516, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=81772, cpu_share=0.01, mem_res=78028, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=89904, cpu_share=0.01, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=184840, cpu_share=0.01, mem_res=177008, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58724, cpu_share=0.01, mem_res=39856, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=16769696, free=7069988, buffers=786384, cached=8039044, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.09, five_min_avg=4.87, fifteen_min_avg=4.11, cpu_share=5.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 02:02:15.660 7f6aef3cc6d8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 02:02:21.273 7f6aef3cc6d8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 02:02:27.932 7f6aeee83438 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 02:02:36.226 7f6aeee83438 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 02:02:56.214 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=52654108, partition_space_available_1k=28436376, percentage_partition_space_used=65, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=114800, cpu_share=0.1, mem_res=104800, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=82352, cpu_share=0.01, mem_res=78156, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=90340, cpu_share=0.01, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=185396, cpu_share=0.01, mem_res=177152, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58744, cpu_share=0.01, mem_res=39864, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=17415292, free=6384388, buffers=787964, cached=8077468, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=11.43, five_min_avg=6.55, fifteen_min_avg=4.72, cpu_share=6.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 02:03:56.206 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=52737248, partition_space_available_1k=28353236, percentage_partition_space_used=65, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=116792, cpu_share=0.08, mem_res=105700, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=82648, cpu_share=0.01, mem_res=78232, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=90544, cpu_share=0.01, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=185620, cpu_share=0.0, mem_res=177168, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58736, cpu_share=0.01, mem_res=39864, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=17913424, free=5835916, buffers=788056, cached=8127716, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.35, five_min_avg=6.63, fifteen_min_avg=4.86, cpu_share=4.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 02:04:56.207 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=52722068, partition_space_available_1k=28368416, percentage_partition_space_used=65, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=120220, cpu_share=0.08, mem_res=107828, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83164, cpu_share=0.01, mem_res=78416, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=90868, cpu_share=0.01, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=185956, cpu_share=0.01, mem_res=177208, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58740, cpu_share=0.01, mem_res=39864, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=17214768, free=6517472, buffers=788084, cached=8144788, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.44, five_min_avg=6.13, fifteen_min_avg=4.8, cpu_share=4.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 02:05:56.235 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=52731700, partition_space_available_1k=28358784, percentage_partition_space_used=65, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=122108, cpu_share=0.05, mem_res=109132, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83304, cpu_share=0.0, mem_res=78416, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=91004, cpu_share=0.0, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=186116, cpu_share=0.0, mem_res=177224, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58740, cpu_share=0.01, mem_res=39864, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=17501424, free=6215848, buffers=788100, cached=8159740, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.79, five_min_avg=5.83, fifteen_min_avg=4.78, cpu_share=4.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 02:06:56.295 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=52741344, partition_space_available_1k=28349140, percentage_partition_space_used=65, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=123316, cpu_share=0.05, mem_res=109676, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83492, cpu_share=0.0, mem_res=78456, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=91148, cpu_share=0.0, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=186324, cpu_share=0.0, mem_res=177260, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58740, cpu_share=0.01, mem_res=39864, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=17375444, free=6321424, buffers=788132, cached=8180112, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=9.46, five_min_avg=6.58, fifteen_min_avg=5.09, cpu_share=5.0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 02:07:34.857 7f6aee793c18 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 02:07:40.654 7f6aee793c18 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 02:07:46.304 7f6af851c0f0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 02:07:54.801 7f6aed1ffeb8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 02:07:56.217 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=52766612, partition_space_available_1k=28323872, percentage_partition_space_used=65, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=123940, cpu_share=0.06, mem_res=109676, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83660, cpu_share=0.0, mem_res=78520, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=91240, cpu_share=0.0, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=186424, cpu_share=0.0, mem_res=177280, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58744, cpu_share=0.01, mem_res=39864, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=17043320, free=6622800, buffers=788792, cached=8210200, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.79, five_min_avg=7.05, fifteen_min_avg=5.35, cpu_share=6.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 02:08:02.339 7f6af2b89c18 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 06/09/2025 02:08:56.226 7f6af16de908 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-544-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=52806808, partition_space_available_1k=28283676, percentage_partition_space_used=65, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=124204, cpu_share=0.01, mem_res=109676, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83760, cpu_share=0.0, mem_res=78524, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=91348, cpu_share=0.0, mem_res=85020, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=186544, cpu_share=0.0, mem_res=177292, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58748, cpu_share=0.01, mem_res=39864, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=16879488, free=6599068, buffers=790516, cached=8396040, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.21, five_min_avg=6.42, fifteen_min_avg=5.25, cpu_share=3.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 06/09/2025 02:08:58.243 7f6af14db780 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map