10/21/2024 02:00:38.785 7fe709ec6f20 [contrail-config-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 10/21/2024 02:00:38.786 7fe709ec6f20 [contrail-config-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 10/21/2024 02:00:38.786 7fe709ec6f20 [contrail-config-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 10/21/2024 02:00:38.796 7fe7036d5248 [contrail-config-nodemgr] [ERROR]: Cannot write http_port 8100 to /tmp/contrail-config-nodemgr.0.http_port 10/21/2024 02:00:38.796 7fe7036d5248 [contrail-config-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8100 10/21/2024 02:00:38.797 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvStart] in state[none] 10/21/2024 02:00:38.798 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:00:38.799 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/21/2024 02:00:38.799 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 10/21/2024 02:00:38.799 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:38.799 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/21/2024 02:00:38.799 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:38.800 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:38.800 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 10/21/2024 02:00:38.800 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:38.891 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'contrail-api', 'group': 'contrail-api', 'pid': 1853705743744759689324036492059398417433751150341213874086244690575054456962, 'start': '1729476036000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-schema', 'group': 'contrail-schema', 'pid': 109844758231913246287481756928139463369416662474740248544389877920308331780226, 'start': '1729476036000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-svc-monitor', 'group': 'contrail-svc-monitor', 'pid': 71797641106728321002455315828516453206439778642619363079725426290091452116487, 'start': '1729476036000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-device-manager', 'group': 'contrail-device-manager', 'pid': 13675773110042261432635878996938833459872379189740033027417278447480400657700, 'start': '1729476036000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-nodemgr', 'group': 'contrail-config-nodemgr', 'pid': 28608217328167142055775636007019106763794812934555161952975043844946031569175, 'start': '1729476036000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 10/21/2024 02:00:39.151 7fe709ec6f20 [contrail-config-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-439-1', _node_type='Config', _instance_id='0', _timestamp=1729476039151183, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 10/21/2024 02:00:39.152 7fe709ec6f20 [contrail-config-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-439-1', _node_type='Config', _instance_id='0', _timestamp=1729476039151938, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-api', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476036000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-schema', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476036000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-svc-monitor', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476036000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-device-manager', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476036000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476036000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 10/21/2024 02:00:39.157 7fe703792648 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:39.163 7fe703792648 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 10/21/2024 02:00:39.164 7fe703792648 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:39.164 7fe703792648 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:39.164 7fe703792648 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/21/2024 02:00:39.164 7fe703792648 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:39.164 7fe703792648 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:39.164 7fe703792648 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 deleted = False process_info = [ << process_name = contrail-api process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476036000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-schema process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476036000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-svc-monitor process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476036000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-device-manager process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476036000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476036000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 10/21/2024 02:00:39.165 7fe703792648 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:42.799 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:00:42.800 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:00:42.801 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:00:42.801 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:00:42.801 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:00:42.801 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:00:42.801 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:00:42.801 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:00:42.801 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:42.802 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:00:42.802 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:42.802 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:42.802 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:00:42.802 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:46.104 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:00:46.801 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:00:46.803 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:00:46.803 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:00:46.803 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:00:46.803 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:00:46.803 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:00:46.803 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:00:46.804 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:00:46.804 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:46.804 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:00:46.804 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:46.804 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:46.804 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:00:46.804 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:48.259 7fe709ec6f20 [contrail-config-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-439-1', _node_type='Config', _instance_id='0', _timestamp=1729476048259237, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-api', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476036000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-schema', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476036000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-svc-monitor', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476036000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-device-manager', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476036000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476036000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 10/21/2024 02:00:48.259 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50859292, partition_space_available_1k=30231192, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1580, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1552, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=6464, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1660, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=69940, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11322236, free=638812, buffers=777464, cached=19926604, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.42, five_min_avg=4.04, fifteen_min_avg=2.82, cpu_share=0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:00:48.260 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:48.260 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 deleted = False process_info = [ << process_name = contrail-api process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476036000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-schema process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476036000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-svc-monitor process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476036000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-device-manager process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476036000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476036000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 10/21/2024 02:00:48.260 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:48.260 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:48.261 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50859292 partition_space_available_1k = 30231192 percentage_partition_space_used = 63 >> } all_core_file_list = [ ] process_mem_cpu_usage = { contrail-api : << mem_virt = 1580 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 1552 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 6464 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 1660 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 69940 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 11322236 free = 638812 buffers = 777464 cached = 19926604 node_type = config-node >> system_cpu_usage = << one_min_avg = 5.42 five_min_avg = 4.04 fifteen_min_avg = 2.82 cpu_share = 0 node_type = config-node >> >> 10/21/2024 02:00:48.261 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:50.805 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:00:50.806 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:00:50.806 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:00:50.806 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:00:50.806 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:00:50.806 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:00:50.806 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:00:50.807 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:00:50.807 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:50.807 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:00:50.807 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:50.807 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:50.807 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:00:50.807 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:54.257 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:00:54.807 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:00:54.808 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:00:54.808 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:00:54.808 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:00:54.808 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:00:54.809 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:00:54.809 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:00:54.809 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:00:54.809 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:54.809 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:00:54.809 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:54.809 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:54.810 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:00:54.810 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:00:58.809 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:01:00.394 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Session Event: TCP Connected 10/21/2024 02:01:00.394 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:01:00.395 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:01:00.395 7fe7036d5448 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:01:00.395 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:01:00.395 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 10/21/2024 02:01:00.395 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 10/21/2024 02:01:00.396 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 10/21/2024 02:01:00.396 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 10/21/2024 02:01:00.396 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 10/21/2024 02:01:00.420 7fe703792648 [contrail-config-nodemgr] [DEBUG]: Extracted sandesh length: 0000001011 10/21/2024 02:01:00.421 7fe703792648 [contrail-config-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 10/21/2024 02:01:00.421 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 10/21/2024 02:01:00.421 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 10/21/2024 02:01:00.422 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:01:00.422 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/21/2024 02:01:00.422 7fe7036d5448 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 10/21/2024 02:01:00.423 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 10/21/2024 02:01:00.424 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 10/21/2024 02:01:09.647 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:01:38.667 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:01:48.390 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51343804, partition_space_available_1k=29746680, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88464, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=67328, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76808, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=164312, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=67772, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14657664, free=6659520, buffers=578332, cached=10769600, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=16.09, five_min_avg=7.33, fifteen_min_avg=4.02, cpu_share=9.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:01:56.008 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:02:16.703 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:02:48.238 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51399332, partition_space_available_1k=29691152, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101440, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70132, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83772, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=163780, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54060, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14543440, free=12132260, buffers=617992, cached=5371424, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=12.52, five_min_avg=7.85, fifteen_min_avg=4.4, cpu_share=7.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:02:50.529 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:03:21.733 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:03:35.885 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:03:48.234 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51440172, partition_space_available_1k=29650312, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102736, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70328, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84144, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166676, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54100, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14619260, free=11615524, buffers=622216, cached=5808116, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.64, five_min_avg=7.04, fifteen_min_avg=4.34, cpu_share=4.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:04:48.238 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51432552, partition_space_available_1k=29657932, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102808, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70320, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84168, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166684, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54100, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14664572, free=11608660, buffers=623828, cached=5768056, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.62, five_min_avg=6.58, fifteen_min_avg=4.35, cpu_share=2.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:04:54.902 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:05:10.162 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:05:24.249 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:05:48.266 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51436928, partition_space_available_1k=29653556, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102884, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70392, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84164, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166744, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54480, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14654764, free=11538604, buffers=626128, cached=5845620, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.07, five_min_avg=5.96, fifteen_min_avg=4.28, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:05:53.147 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:06:48.260 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51449680, partition_space_available_1k=29640804, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103012, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70428, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84220, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166804, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54484, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14637824, free=10248764, buffers=678360, cached=7100168, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=7.52, five_min_avg=6.79, fifteen_min_avg=4.68, cpu_share=5.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:07:48.276 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51453452, partition_space_available_1k=29637032, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103204, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70424, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84244, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166812, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54500, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14649404, free=10231588, buffers=678364, cached=7105760, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.95, five_min_avg=5.88, fifteen_min_avg=4.49, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:08:48.282 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51457100, partition_space_available_1k=29633384, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103256, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70480, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84248, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166848, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54524, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14652568, free=10222980, buffers=678372, cached=7111196, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.66, five_min_avg=5.18, fifteen_min_avg=4.34, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:08:58.269 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:09:12.162 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:09:29.166 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:09:48.288 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51465344, partition_space_available_1k=29625140, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103328, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70508, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84296, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166912, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54536, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14642088, free=10227440, buffers=678464, cached=7117124, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.24, five_min_avg=4.6, fifteen_min_avg=4.19, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:09:53.990 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:10:48.305 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51465176, partition_space_available_1k=29625308, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103392, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70500, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84324, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166920, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54552, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14637936, free=10226384, buffers=678476, cached=7122320, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.56, five_min_avg=4.32, fifteen_min_avg=4.12, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:11:48.340 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51469316, partition_space_available_1k=29621168, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103396, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70556, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84316, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166960, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54556, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=15044924, free=9808236, buffers=678476, cached=7133480, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.64, five_min_avg=5.3, fifteen_min_avg=4.45, cpu_share=3.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:12:48.307 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51480256, partition_space_available_1k=29610228, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103436, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70588, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84380, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167024, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54572, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14705924, free=10130716, buffers=678484, cached=7149992, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.33, five_min_avg=5.03, fifteen_min_avg=4.41, cpu_share=3.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:13:09.164 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:13:22.989 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:13:40.138 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:13:48.316 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51485740, partition_space_available_1k=29604744, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103528, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70588, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84412, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167028, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54576, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14808344, free=10018632, buffers=678492, cached=7159648, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.45, five_min_avg=4.55, fifteen_min_avg=4.28, cpu_share=3.0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:14:48.309 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51488360, partition_space_available_1k=29602124, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103548, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70640, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84404, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167068, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54576, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14774460, free=10048272, buffers=678524, cached=7163860, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.38, five_min_avg=4.06, fifteen_min_avg=4.13, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:15:26.119 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:15:48.330 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51492040, partition_space_available_1k=29598444, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103624, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70680, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84460, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167136, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54576, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14808208, free=10009016, buffers=678528, cached=7169364, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.22, five_min_avg=3.69, fifteen_min_avg=4.0, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:16:48.320 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51495952, partition_space_available_1k=29594532, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103776, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70672, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84484, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167140, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54588, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14788456, free=10022744, buffers=678536, cached=7175380, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.18, five_min_avg=3.41, fifteen_min_avg=3.88, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:17:19.355 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:17:34.685 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:17:48.555 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51509672, partition_space_available_1k=29580812, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=103960, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70732, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84480, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167176, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54580, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14878452, free=9899636, buffers=679500, cached=7207528, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.77, five_min_avg=3.9, fifteen_min_avg=4.01, cpu_share=5.0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:17:52.002 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:18:48.322 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51511240, partition_space_available_1k=29579244, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104080, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70760, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84532, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167240, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54588, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14815248, free=9953776, buffers=679504, cached=7216588, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.17, five_min_avg=3.6, fifteen_min_avg=3.9, cpu_share=3.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:19:48.320 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51514728, partition_space_available_1k=29575756, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104148, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70764, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84564, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167244, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54588, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14813568, free=9949204, buffers=679616, cached=7222728, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.08, five_min_avg=3.22, fifteen_min_avg=3.75, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:20:48.337 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51518264, partition_space_available_1k=29572220, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104152, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70820, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84556, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167284, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54588, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14821780, free=9936360, buffers=679620, cached=7227356, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.08, five_min_avg=3.02, fifteen_min_avg=3.65, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:21:14.354 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:21:48.322 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51522420, partition_space_available_1k=29568064, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104196, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70852, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84612, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167344, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54588, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14820184, free=9932796, buffers=679692, cached=7232444, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.52, five_min_avg=3.0, fifteen_min_avg=3.6, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:22:48.338 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51526072, partition_space_available_1k=29564412, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104256, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70844, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84636, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167356, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54608, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14871368, free=9876452, buffers=679820, cached=7237476, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.4, five_min_avg=2.89, fifteen_min_avg=3.53, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:22:51.422 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:23:05.145 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:23:28.957 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:23:48.349 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51540920, partition_space_available_1k=29549564, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104264, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70908, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84644, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167392, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54640, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14900328, free=9824800, buffers=679832, cached=7260156, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.19, five_min_avg=3.62, fifteen_min_avg=3.74, cpu_share=5.0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:24:48.355 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51540712, partition_space_available_1k=29549772, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104304, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70936, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84700, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167456, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54640, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14859876, free=9855292, buffers=679840, cached=7270108, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.74, five_min_avg=3.21, fifteen_min_avg=3.59, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:25:48.375 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51544648, partition_space_available_1k=29545836, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104368, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70932, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84692, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167464, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54644, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14848832, free=9861016, buffers=679848, cached=7275420, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.31, five_min_avg=3.32, fifteen_min_avg=3.6, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:26:48.378 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51548636, partition_space_available_1k=29541848, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104376, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70984, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84720, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167500, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54644, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14853420, free=9851696, buffers=679848, cached=7280152, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.13, five_min_avg=2.96, fifteen_min_avg=3.46, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:27:07.114 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:27:44.837 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:27:48.394 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51547888, partition_space_available_1k=29542596, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104464, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71012, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84772, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167564, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54648, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14906312, free=9794556, buffers=679852, cached=7284396, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.92, five_min_avg=2.74, fifteen_min_avg=3.35, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:27:58.605 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:28:17.345 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:28:48.403 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51559200, partition_space_available_1k=29531284, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104536, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71008, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84768, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167568, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54652, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14878436, free=9805640, buffers=679852, cached=7301188, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.62, five_min_avg=3.89, fifteen_min_avg=3.72, cpu_share=5.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:29:48.409 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51562972, partition_space_available_1k=29527512, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104556, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71060, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84800, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167612, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54652, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14874312, free=9809968, buffers=679912, cached=7300924, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.72, five_min_avg=3.62, fifteen_min_avg=3.64, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:30:48.425 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51561896, partition_space_available_1k=29528588, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104604, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71088, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84852, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167672, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54660, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14901168, free=9777928, buffers=679912, cached=7306108, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.37, five_min_avg=3.5, fifteen_min_avg=3.59, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:31:44.318 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:31:48.417 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51566112, partition_space_available_1k=29524372, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104660, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71084, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84844, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167676, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54660, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=15001156, free=9669292, buffers=679916, cached=7314752, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.29, five_min_avg=3.42, fifteen_min_avg=3.55, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:32:25.938 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:32:39.906 7fe7036d5448 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:32:48.429 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51568636, partition_space_available_1k=29521848, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104672, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71136, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84872, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167744, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54668, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14986576, free=9683788, buffers=679916, cached=7314836, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.56, five_min_avg=3.16, fifteen_min_avg=3.45, cpu_share=2.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:32:56.923 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:33:49.271 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51575268, partition_space_available_1k=29515216, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104708, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71164, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84924, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167784, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54660, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14952052, free=9791896, buffers=679920, cached=7241248, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.43, five_min_avg=3.76, fifteen_min_avg=3.63, cpu_share=5.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:34:48.437 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51575196, partition_space_available_1k=29515288, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104776, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71160, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84924, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167788, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54668, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14953552, free=9790244, buffers=679920, cached=7241400, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.68, five_min_avg=3.6, fifteen_min_avg=3.59, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:35:18.873 7fe703792648 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connection Closed 10/21/2024 02:35:18.873 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 10/21/2024 02:35:23.879 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:35:23.879 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 10/21/2024 02:35:23.879 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:23.879 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 10/21/2024 02:35:23.879 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:23.879 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:35:23.880 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:35:23.880 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:23.880 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:35:23.880 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:23.880 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:23.880 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:35:23.881 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:27.881 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:35:32.886 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:35:32.886 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:35:32.886 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:32.886 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:35:32.887 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:32.887 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:35:32.887 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:35:32.887 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:32.887 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:35:32.887 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:32.888 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:32.888 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:35:32.888 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:36.889 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:35:41.891 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:35:41.891 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:35:41.892 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:41.892 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:35:41.892 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:41.892 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:35:41.893 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:35:41.893 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:41.893 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:35:41.893 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:41.894 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:41.894 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:35:41.894 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:45.892 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:35:48.449 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51579436, partition_space_available_1k=29511048, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104780, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71212, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84956, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167852, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54676, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14944380, free=9794372, buffers=679920, cached=7246444, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.55, five_min_avg=3.28, fifteen_min_avg=3.47, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:35:50.895 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:35:50.896 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:35:50.896 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:50.896 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:35:50.896 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:50.896 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:50.896 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51579436 partition_space_available_1k = 29511048 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 104780 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 71212 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 84956 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 167852 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 54676 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14944380 free = 9794372 buffers = 679920 cached = 7246444 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.55 five_min_avg = 3.28 fifteen_min_avg = 3.47 cpu_share = 2.75 node_type = config-node >> >> 10/21/2024 02:35:50.896 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:50.897 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:35:50.897 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:35:50.897 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:50.897 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:35:50.898 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:50.898 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:50.898 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:35:50.898 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:54.898 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:35:58.194 7fe703792648 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:35:59.901 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:35:59.901 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:35:59.901 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:59.902 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:35:59.902 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:59.902 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:35:59.902 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:35:59.902 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:59.902 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:35:59.903 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:59.903 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:59.903 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:35:59.903 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:03.908 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:36:08.916 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:36:08.916 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:36:08.916 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:08.916 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:36:08.916 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:08.917 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:36:08.917 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:36:08.917 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:08.917 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:36:08.917 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:08.917 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:08.917 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:36:08.918 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:12.919 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:36:17.923 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:36:17.923 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:36:17.923 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:17.924 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:36:17.924 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:17.924 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:36:17.924 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:36:17.925 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:17.925 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:36:17.925 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:17.925 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:17.925 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:36:17.925 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:21.925 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:36:26.929 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:36:26.929 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:36:26.929 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:26.929 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:36:26.929 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:26.929 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:36:26.930 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:36:26.930 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:26.930 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:36:26.930 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:26.930 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:26.930 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:36:26.930 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:30.932 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:36:35.934 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:36:35.935 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:36:35.935 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:35.935 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:36:35.935 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:35.935 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:36:35.935 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:36:35.936 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:35.936 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:36:35.936 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:35.936 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:35.936 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:36:35.936 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:39.935 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:36:44.937 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:36:44.937 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:36:44.937 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:44.937 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:36:44.938 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:44.938 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:36:44.938 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:36:44.938 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:44.938 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:36:44.938 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:44.938 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:44.939 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:36:44.939 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:48.439 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51583740, partition_space_available_1k=29506744, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104844, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71240, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85012, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167892, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54696, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14963360, free=9775184, buffers=679924, cached=7246648, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.2, five_min_avg=3.04, fifteen_min_avg=3.38, cpu_share=2.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:36:48.440 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:48.440 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51583740 partition_space_available_1k = 29506744 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 104844 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 71240 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 85012 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 167892 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 54696 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14963360 free = 9775184 buffers = 679924 cached = 7246648 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.2 five_min_avg = 3.04 fifteen_min_avg = 3.38 cpu_share = 2.88 node_type = config-node >> >> 10/21/2024 02:36:48.440 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:48.937 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:36:53.942 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:36:53.942 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:36:53.942 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:53.942 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:36:53.942 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:53.942 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:36:53.943 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:36:53.943 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:53.943 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:36:53.943 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:53.943 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:53.943 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:36:53.944 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:57.945 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:02.950 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:37:02.950 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:37:02.950 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:02.950 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:37:02.950 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:02.950 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:37:02.951 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:37:02.951 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:02.952 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:37:02.952 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:02.952 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:02.952 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:37:02.952 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:06.951 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:11.956 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:37:11.956 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:37:11.957 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:11.957 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:37:11.957 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:11.957 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:37:11.958 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:37:11.958 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:11.958 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:37:11.958 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:11.958 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:11.959 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:37:11.959 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:15.958 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:20.960 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:37:20.960 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:37:20.960 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:20.961 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:37:20.961 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:20.961 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:37:20.961 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:37:20.961 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:20.962 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:37:20.962 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:20.962 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:20.962 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:37:20.962 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:24.961 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:29.965 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:37:29.965 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:37:29.965 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:29.965 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:37:29.965 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:29.965 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:37:29.966 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:37:29.966 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:29.966 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:37:29.966 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:29.966 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:29.966 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:37:29.967 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:33.967 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:38.972 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:37:38.972 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:37:38.972 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:38.972 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:37:38.972 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:38.972 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:37:38.973 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:37:38.973 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:38.973 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:37:38.973 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:38.973 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:38.973 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:37:38.974 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:42.972 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:47.975 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:37:47.975 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:37:47.975 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:47.975 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:37:47.975 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:47.975 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:37:47.976 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:37:47.976 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:47.976 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:37:47.976 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:47.976 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:47.976 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:37:47.977 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:48.440 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51582988, partition_space_available_1k=29507496, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104900, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71232, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85008, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167896, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54732, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14930888, free=9802564, buffers=679924, cached=7251740, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.27, five_min_avg=2.9, fifteen_min_avg=3.31, cpu_share=2.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:37:48.440 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:48.441 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51582988 partition_space_available_1k = 29507496 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 104900 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 71232 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 85008 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 167896 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 54732 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14930888 free = 9802564 buffers = 679924 cached = 7251740 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.27 five_min_avg = 2.9 fifteen_min_avg = 3.31 cpu_share = 2.88 node_type = config-node >> >> 10/21/2024 02:37:48.441 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:51.976 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:56.980 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:37:56.980 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:37:56.980 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:56.980 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:37:56.980 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:56.980 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:37:56.981 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:37:56.981 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:56.981 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:37:56.981 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:56.981 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:56.981 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:37:56.981 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:59.413 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:38:00.981 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:38:05.983 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:38:05.983 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:38:05.983 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:05.983 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:38:05.984 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:05.984 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:38:05.984 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:38:05.985 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:05.985 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:38:05.985 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:05.985 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:05.985 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:38:05.985 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:09.986 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:38:13.429 7fe703792648 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:38:14.989 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:38:14.989 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:38:14.989 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:14.990 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:38:14.990 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:14.990 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:38:14.991 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:38:14.991 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:14.991 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:38:14.991 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:14.991 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:14.991 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:38:14.991 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:18.993 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:38:23.997 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:38:23.997 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:38:23.998 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:23.998 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:38:23.998 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:23.998 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:38:23.999 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:38:23.999 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:23.999 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:38:23.999 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:23.999 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:23.999 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:38:23.999 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:28.001 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:38:30.794 7fe703792648 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:38:33.004 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:38:33.005 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:38:33.005 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:33.005 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:38:33.005 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:33.005 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:38:33.006 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:38:33.006 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:33.006 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:38:33.006 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:33.006 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:33.007 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:38:33.007 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:37.005 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:38:42.007 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:38:42.008 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:38:42.008 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:42.008 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:38:42.008 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:42.008 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:38:42.009 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:38:42.009 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:42.009 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:38:42.009 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:42.009 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:42.010 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:38:42.010 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:46.008 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:38:48.445 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51591332, partition_space_available_1k=29499152, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104908, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71288, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85032, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167960, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54784, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14977572, free=9748448, buffers=679924, cached=7259172, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.78, five_min_avg=2.9, fifteen_min_avg=3.28, cpu_share=3.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:38:51.010 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:38:51.011 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:38:51.011 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:51.011 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:38:51.011 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:51.011 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:51.011 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51591332 partition_space_available_1k = 29499152 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 104908 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 71288 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 85032 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 167960 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 54784 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14977572 free = 9748448 buffers = 679924 cached = 7259172 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.78 five_min_avg = 2.9 fifteen_min_avg = 3.28 cpu_share = 3.25 node_type = config-node >> >> 10/21/2024 02:38:51.011 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:51.011 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:38:51.012 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:38:51.012 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:51.012 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:38:51.012 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:51.012 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:51.012 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:38:51.012 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:55.013 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:00.017 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:00.017 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:00.017 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:00.018 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:00.018 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:00.018 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:00.019 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:00.019 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:00.019 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:00.019 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:00.019 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:00.019 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:39:00.020 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:04.020 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:09.022 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:09.022 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:09.022 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:09.022 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:09.022 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:09.023 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:09.023 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:09.023 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:09.023 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:09.023 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:09.023 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:09.023 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:39:09.024 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:13.025 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:18.029 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:18.030 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:18.030 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:18.030 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:18.030 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:18.030 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:18.031 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:18.031 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:18.031 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:18.031 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:18.032 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:18.032 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:39:18.032 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:22.030 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:27.035 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:27.036 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:27.036 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:27.036 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:27.036 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:27.036 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:27.037 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:27.037 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:27.037 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:27.037 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:27.038 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:27.038 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:39:27.038 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:31.037 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:36.042 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:36.043 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:36.043 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:36.043 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:36.043 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:36.043 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:36.044 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:36.044 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:36.044 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:36.044 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:36.044 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:36.044 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:39:36.044 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:40.043 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:45.045 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:45.046 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:45.046 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:45.046 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:45.046 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:45.046 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:45.047 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:45.047 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:45.047 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:45.047 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:45.047 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:45.047 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:39:45.048 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:48.440 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51593052, partition_space_available_1k=29497432, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104996, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71316, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85164, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=167996, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54800, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14984844, free=9725380, buffers=679924, cached=7274968, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.71, five_min_avg=3.45, fifteen_min_avg=3.44, cpu_share=5.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:39:48.440 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:48.441 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51593052 partition_space_available_1k = 29497432 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 104996 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 71316 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 85164 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 167996 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 54800 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14984844 free = 9725380 buffers = 679924 cached = 7274968 node_type = config-node >> system_cpu_usage = << one_min_avg = 4.71 five_min_avg = 3.45 fifteen_min_avg = 3.44 cpu_share = 5.12 node_type = config-node >> >> 10/21/2024 02:39:48.441 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:49.046 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:54.048 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:54.049 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:54.049 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:54.049 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:54.049 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:54.049 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:54.050 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:54.050 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:54.050 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:54.050 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:54.050 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:54.050 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:39:54.050 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:58.051 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:03.057 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:03.057 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:03.057 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:03.058 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:03.058 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:03.058 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:03.058 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:03.058 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:03.058 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:03.058 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:03.059 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:03.059 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:40:03.059 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:07.058 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:12.064 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:12.064 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:12.064 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:12.064 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:12.064 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:12.064 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:12.065 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:12.065 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:12.065 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:12.065 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:12.065 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:12.065 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:40:12.065 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:16.065 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:21.070 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:21.070 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:21.070 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:21.071 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:21.071 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:21.071 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:21.072 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:21.072 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:21.072 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:21.072 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:21.072 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:21.072 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:40:21.073 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:24.972 7fe7036d5548 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:40:25.071 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:30.076 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:30.076 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:30.076 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:30.076 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:30.076 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:30.076 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:30.077 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:30.077 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:30.077 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:30.077 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:30.077 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:30.077 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:40:30.077 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:34.079 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:39.081 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:39.082 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:39.082 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:39.082 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:39.082 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:39.082 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:39.083 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:39.083 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:39.083 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:39.083 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:39.083 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:39.083 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:40:39.083 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:43.082 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:48.085 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:48.085 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:48.085 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:48.086 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:48.086 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:48.086 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:48.087 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:48.087 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:48.087 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:48.087 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:48.087 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:48.087 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:40:48.087 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:48.469 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51597972, partition_space_available_1k=29492512, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=105032, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71312, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85160, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168000, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54828, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14993324, free=9690156, buffers=681916, cached=7299720, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.97, five_min_avg=3.16, fifteen_min_avg=3.34, cpu_share=2.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:40:48.470 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:48.470 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51597972 partition_space_available_1k = 29492512 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 105032 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 71312 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 85160 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 168000 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 54828 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14993324 free = 9690156 buffers = 681916 cached = 7299720 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.97 five_min_avg = 3.16 fifteen_min_avg = 3.34 cpu_share = 2.88 node_type = config-node >> >> 10/21/2024 02:40:48.470 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:52.085 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:57.091 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:57.092 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:57.092 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:57.092 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:57.092 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:57.092 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:57.093 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:57.093 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:57.093 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:57.094 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:57.094 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:57.094 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:40:57.094 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:01.094 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:06.098 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:06.098 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:06.098 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:06.098 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:06.098 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:06.098 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:06.099 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:06.099 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:06.099 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:06.099 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:06.099 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:06.099 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:41:06.100 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:10.099 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:15.104 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:15.105 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:15.105 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:15.105 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:15.105 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:15.105 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:15.106 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:15.106 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:15.106 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:15.106 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:15.106 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:15.106 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:41:15.106 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:19.107 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:24.110 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:24.110 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:24.110 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:24.110 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:24.111 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:24.111 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:24.111 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:24.112 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:24.112 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:24.112 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:24.112 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:24.112 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:41:24.112 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:28.111 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:33.117 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:33.117 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:33.118 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:33.118 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:33.118 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:33.118 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:33.119 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:33.119 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:33.119 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:33.119 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:33.119 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:33.119 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:41:33.120 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:37.121 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:42.124 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:42.124 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:42.124 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:42.124 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:42.125 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:42.125 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:42.125 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:42.126 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:42.126 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:42.126 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:42.126 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:42.126 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:41:42.126 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:46.125 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:48.448 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51596696, partition_space_available_1k=29493788, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=105040, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71368, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85188, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168072, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54852, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14983620, free=9700616, buffers=681916, cached=7298964, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.46, five_min_avg=2.94, fifteen_min_avg=3.25, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:41:51.129 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:51.129 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:51.129 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:51.130 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:51.130 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:51.130 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:51.130 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51596696 partition_space_available_1k = 29493788 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 105040 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 71368 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 85188 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 168072 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 54852 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14983620 free = 9700616 buffers = 681916 cached = 7298964 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.46 five_min_avg = 2.94 fifteen_min_avg = 3.25 cpu_share = 2.75 node_type = config-node >> >> 10/21/2024 02:41:51.130 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:51.130 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:51.131 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:51.131 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:51.131 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:51.132 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:51.132 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:51.132 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:41:51.132 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:55.131 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:00.137 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:00.137 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:00.137 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:00.137 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:00.137 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:00.137 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:00.138 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:00.138 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:00.138 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:00.138 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:00.138 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:00.138 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:42:00.138 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:04.139 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:09.141 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:09.141 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:09.141 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:09.141 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:09.142 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:09.142 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:09.142 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:09.143 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:09.143 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:09.143 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:09.144 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:09.144 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:42:09.144 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:13.143 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:18.148 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:18.149 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:18.149 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:18.149 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:18.149 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:18.149 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:18.150 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:18.150 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:18.150 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:18.150 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:18.150 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:18.150 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:42:18.150 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:22.151 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:27.156 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:27.156 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:27.156 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:27.157 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:27.157 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:27.157 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:27.158 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:27.158 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:27.158 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:27.158 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:27.158 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:27.159 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:42:27.159 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:31.159 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:36.165 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:36.165 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:36.165 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:36.165 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:36.165 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:36.165 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:36.166 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:36.166 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:36.166 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:36.166 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:36.166 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:36.166 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:42:36.166 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:40.166 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:45.169 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:45.169 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:45.169 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:45.169 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:45.170 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:45.170 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:45.170 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:45.171 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:45.171 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:45.171 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:45.171 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:45.171 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:42:45.171 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:48.449 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51600672, partition_space_available_1k=29489812, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=105104, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71392, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85240, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168108, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54872, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14974980, free=9704656, buffers=681916, cached=7303564, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.49, five_min_avg=2.82, fifteen_min_avg=3.19, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:42:48.449 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:48.449 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51600672 partition_space_available_1k = 29489812 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 105104 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 71392 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 85240 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 168108 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 54872 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14974980 free = 9704656 buffers = 681916 cached = 7303564 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.49 five_min_avg = 2.82 fifteen_min_avg = 3.19 cpu_share = 2.75 node_type = config-node >> >> 10/21/2024 02:42:48.449 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:49.168 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:54.170 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:54.170 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:54.170 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:54.171 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:54.171 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:54.171 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:54.171 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:54.171 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:54.172 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:54.172 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:54.172 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:54.172 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:42:54.172 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:58.173 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:03.177 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:03.177 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:03.177 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:03.177 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:03.177 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:03.178 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:03.178 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:03.178 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:03.178 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:03.178 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:03.179 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:03.179 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:43:03.179 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:07.180 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:12.184 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:12.184 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:12.185 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:12.185 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:12.185 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:12.185 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:12.185 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:12.186 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:12.186 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:12.186 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:12.186 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:12.186 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:43:12.186 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:16.187 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:21.192 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:21.193 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:21.193 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:21.193 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:21.193 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:21.193 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:21.194 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:21.194 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:21.194 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:21.194 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:21.194 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:21.194 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:43:21.194 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:25.193 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:30.197 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:30.197 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:30.197 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:30.198 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:30.198 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:30.198 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:30.199 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:30.199 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:30.199 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:30.199 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:30.199 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:30.199 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:43:30.200 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:34.198 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:39.201 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:39.201 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:39.201 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:39.201 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:39.201 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:39.201 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:39.202 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:39.202 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:39.202 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:39.202 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:39.202 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:39.202 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:43:39.202 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:43.201 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:48.204 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:48.204 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:48.204 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:48.204 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:48.204 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:48.205 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:48.205 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:48.205 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:48.206 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:48.206 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:48.206 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:48.206 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:43:48.206 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:48.450 7fe709ec6f20 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51599724, partition_space_available_1k=29490760, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=105144, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71384, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85240, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168112, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54896, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14953912, free=9706544, buffers=682788, cached=7321872, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.94, five_min_avg=2.61, fifteen_min_avg=3.09, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 10/21/2024 02:43:48.450 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:48.451 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51599724 partition_space_available_1k = 29490760 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 105144 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 71384 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 85240 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 168112 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 54896 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14953912 free = 9706544 buffers = 682788 cached = 7321872 node_type = config-node >> system_cpu_usage = << one_min_avg = 1.94 five_min_avg = 2.61 fifteen_min_avg = 3.09 cpu_share = 2.75 node_type = config-node >> >> 10/21/2024 02:43:48.451 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:52.206 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:55.829 7fe703792648 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:43:56.849 7fe703792648 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/21/2024 02:43:57.209 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:57.209 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:57.209 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:57.209 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:57.209 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:57.209 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:57.210 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:57.210 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:57.210 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:57.210 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:57.210 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:57.210 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:43:57.210 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:44:01.209 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:44:06.214 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:44:06.215 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:44:06.215 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:44:06.215 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:44:06.215 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:44:06.215 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:44:06.216 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:44:06.216 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:44:06.216 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:44:06.217 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:44:06.217 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:44:06.217 7fe7036d5548 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1729476038799110 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 10/21/2024 02:44:06.217 7fe7036d5548 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]