12/23/2024 01:51:26.555 7fbd1299f348 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 12/23/2024 01:51:26.555 7fbd1299f348 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 12/23/2024 01:51:26.555 7fbd1299f348 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 12/23/2024 01:51:26.565 7fbd10094390 [contrail-vrouter-nodemgr] [ERROR]: Cannot write http_port 8102 to /tmp/contrail-vrouter-nodemgr.0.http_port 12/23/2024 01:51:26.566 7fbd10094390 [contrail-vrouter-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8102 12/23/2024 01:51:26.566 7fbd10098cc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvStart] in state[none] 12/23/2024 01:51:26.567 7fbd10098cc0 [contrail-vrouter-nodemgr] [DEBUG]: Add uve in the [ObjectVRouter:NodeStatusUVE] map 12/23/2024 01:51:26.567 7fbd10098cc0 [contrail-vrouter-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 12/23/2024 01:51:26.568 7fbd10098cc0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 12/23/2024 01:51:26.568 7fbd10098cc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:51:26.568 7fbd10098cc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 12/23/2024 01:51:26.568 7fbd10098cc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:51:26.568 7fbd10098cc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:51:26.568 7fbd10098cc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1734918686567745 collector_name = collector_ip = collector_list = [ 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 12/23/2024 01:51:26.568 7fbd10098cc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:51:26.621 7fbd1299f348 [contrail-vrouter-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'contrail-vrouter-agent', 'group': 'contrail-vrouter-agent', 'pid': 78786993785516675965502949264807261932177893255208899139405783533101954226484, 'start': '1734918682000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-vrouter-nodemgr', 'group': 'contrail-vrouter-nodemgr', 'pid': 40244814571361394398235453171305342697750718238918895628694051712392230834693, 'start': '1734918624000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 12/23/2024 01:51:26.747 7fbd1299f348 [contrail-vrouter-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='an-jenkins-deploy-platform-juju-k8s-882-1', _node_type='Compute', _instance_id='0', _timestamp=1734918686747354, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-vrouter-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 12/23/2024 01:51:26.747 7fbd1299f348 [contrail-vrouter-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='an-jenkins-deploy-platform-juju-k8s-882-1', _node_type='Compute', _instance_id='0', _timestamp=1734918686747824, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-vrouter-agent', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1734918682000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-vrouter-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1734918624000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-12-23 01:03:26.567380", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7088-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 12/23/2024 01:51:26.750 7fbd1027bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:51:26.750 7fbd1027bd68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1 build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-12-23 01:03:26.567380", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7088-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 2 num_core_per_socket = 2 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 12/23/2024 01:51:26.750 7fbd1027bd68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:51:26.750 7fbd1027bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:51:26.751 7fbd1027bd68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 12/23/2024 01:51:26.751 7fbd1027bd68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:51:26.751 7fbd1027bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:51:26.751 7fbd1027bd68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1 deleted = False process_info = [ << process_name = contrail-vrouter-agent process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1734918682000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-vrouter-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1734918624000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-12-23 01:03:26.567380", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7088-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 12/23/2024 01:51:26.751 7fbd1027bd68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:51:29.770 7fbd1299f348 [contrail-vrouter-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='an-jenkins-deploy-platform-juju-k8s-882-1', _node_type='Compute', _instance_id='0', _timestamp=1734918689770156, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-vrouter-agent', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1734918682000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-vrouter-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1734918624000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-12-23 01:03:26.567380", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7088-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 12/23/2024 01:51:29.770 7fbd1299f348 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6801388, partition_space_available_1k=43802444, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=246836, cpu_share=0, mem_res=108120, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=73772, cpu_share=0, mem_res=38336, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1156292, free=5089340, buffers=211668, cached=1482152, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.58, five_min_avg=0.29, fifteen_min_avg=0.2, cpu_share=0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 01:51:29.770 7fbd10561780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:51:29.770 7fbd10561780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1 deleted = False process_info = [ << process_name = contrail-vrouter-agent process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1734918682000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-vrouter-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1734918624000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-12-23 01:03:26.567380", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7088-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 12/23/2024 01:51:29.771 7fbd10561780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:51:29.771 7fbd10561780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:51:29.771 7fbd10561780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6801388 partition_space_available_1k = 43802444 percentage_partition_space_used = 13 >> } all_core_file_list = [ ] process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 246836 cpu_share = 0 mem_res = 108120 >>contrail-vrouter-nodemgr : << mem_virt = 73772 cpu_share = 0 mem_res = 38336 >> } system_mem_usage = << total = 7939452 used = 1156292 free = 5089340 buffers = 211668 cached = 1482152 node_type = vrouter >> system_cpu_usage = << one_min_avg = 0.58 five_min_avg = 0.29 fifteen_min_avg = 0.2 cpu_share = 0 node_type = vrouter >> >> 12/23/2024 01:51:29.771 7fbd10561780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:51:30.568 7fbd12846240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/23/2024 01:51:30.570 7fbd12846240 [contrail-vrouter-nodemgr] [INFO]: Session Event: TCP Connected 12/23/2024 01:51:30.570 7fbd12846240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/23/2024 01:51:30.570 7fbd12846240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/23/2024 01:51:30.571 7fbd12846240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/23/2024 01:51:30.571 7fbd12846240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/23/2024 01:51:30.571 7fbd12846240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 12/23/2024 01:51:30.571 7fbd12846240 [contrail-vrouter-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 12/23/2024 01:51:30.571 7fbd12846240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 12/23/2024 01:51:30.572 7fbd12846240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 12/23/2024 01:51:30.572 7fbd12846240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 12/23/2024 01:51:30.576 7fbd10098a20 [contrail-vrouter-nodemgr] [DEBUG]: Extracted sandesh length: 0000001013 12/23/2024 01:51:30.577 7fbd10098a20 [contrail-vrouter-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 12/23/2024 01:51:30.577 7fbd128a0d68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 12/23/2024 01:51:30.577 7fbd128a0d68 [contrail-vrouter-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 12/23/2024 01:51:30.577 7fbd128a0d68 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 12/23/2024 01:51:30.578 7fbd128a0d68 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 12/23/2024 01:51:30.578 7fbd128a0d68 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 12/23/2024 01:51:30.578 7fbd12846240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 12/23/2024 01:51:30.580 7fbd12846240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 12/23/2024 01:52:29.780 7fbd1299f348 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6801440, partition_space_available_1k=43802392, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249204, cpu_share=0.5, mem_res=110160, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=73964, cpu_share=0.0, mem_res=38424, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1137728, free=5095584, buffers=211696, cached=1494444, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.25, five_min_avg=0.25, fifteen_min_avg=0.19, cpu_share=0.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 01:53:29.795 7fbd1299f348 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6801472, partition_space_available_1k=43802360, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=250632, cpu_share=0.5, mem_res=111480, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=73900, cpu_share=0.0, mem_res=38476, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1136228, free=5094472, buffers=211748, cached=1497004, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.12, five_min_avg=0.22, fifteen_min_avg=0.18, cpu_share=0.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 01:53:42.052 7fbd100afeb8 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_process_state:contrail-vrouter-agent,group:contrail-vrouter-agent,state:PROCESS_STATE_RUNNING 12/23/2024 01:53:42.053 7fbd100afeb8 [contrail-vrouter-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='an-jenkins-deploy-platform-juju-k8s-882-1', _node_type='Compute', _instance_id='0', _timestamp=1734918822053101, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-vrouter-agent', process_state='PROCESS_STATE_RUNNING', start_count=2, stop_count=0, exit_count=0, last_start_time='1734918822053022', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-vrouter-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1734918624000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-12-23 01:03:26.567380", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7088-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 12/23/2024 01:53:42.077 7fbd100afeb8 [contrail-vrouter-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'contrail-vrouter-agent', 'group': 'contrail-vrouter-agent', 'pid': 99196369161329632392009110066709191903018524786806242401732654861824482163388, 'start': '1734918816000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-vrouter-nodemgr', 'group': 'contrail-vrouter-nodemgr', 'pid': 40244814571361394398235453171305342697750718238918895628694051712392230834693, 'start': '1734918624000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 12/23/2024 01:53:42.078 7fbd100afeb8 [contrail-vrouter-nodemgr] [DEBUG]: DBG: update_current_processes: process_state_db='{'default': {'contrail-vrouter-agent': , 'contrail-vrouter-nodemgr': }}' 12/23/2024 01:53:46.048 7f8c8f9d5748 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 12/23/2024 01:53:46.048 7f8c8f9d5748 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 12/23/2024 01:53:46.048 7f8c8f9d5748 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 12/23/2024 01:53:46.057 7f8c8d23f390 [contrail-vrouter-nodemgr] [ERROR]: Cannot write http_port 8102 to /tmp/contrail-vrouter-nodemgr.0.http_port 12/23/2024 01:53:46.058 7f8c8d23f390 [contrail-vrouter-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8102 12/23/2024 01:53:46.059 7f8c8d243cc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvStart] in state[none] 12/23/2024 01:53:46.060 7f8c8d243cc0 [contrail-vrouter-nodemgr] [DEBUG]: Add uve in the [ObjectVRouter:NodeStatusUVE] map 12/23/2024 01:53:46.061 7f8c8d243cc0 [contrail-vrouter-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 12/23/2024 01:53:46.061 7f8c8d243cc0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 12/23/2024 01:53:46.061 7f8c8d243cc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:53:46.063 7f8c8d243cc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 12/23/2024 01:53:46.063 7f8c8d243cc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:53:46.063 7f8c8d243cc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:53:46.063 7f8c8d243cc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1734918826061119 collector_name = collector_ip = collector_list = [ 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 12/23/2024 01:53:46.063 7f8c8d243cc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:53:46.105 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'contrail-vrouter-agent', 'group': 'contrail-vrouter-agent', 'pid': 99196369161329632392009110066709191903018524786806242401732654861824482163388, 'start': '1734918816000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-vrouter-nodemgr', 'group': 'contrail-vrouter-nodemgr', 'pid': 50693436604786027655479288070899891561253687128363257974562368157976391347300, 'start': '1734918823000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 12/23/2024 01:53:46.166 7f8c8db43ac8 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 12/23/2024 01:53:46.277 7f8c8f9d5748 [contrail-vrouter-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='an-jenkins-deploy-platform-juju-k8s-882-1', _node_type='Compute', _instance_id='0', _timestamp=1734918826277416, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-vrouter-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 12/23/2024 01:53:46.278 7f8c8f9d5748 [contrail-vrouter-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='an-jenkins-deploy-platform-juju-k8s-882-1', _node_type='Compute', _instance_id='0', _timestamp=1734918826277996, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-vrouter-agent', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1734918816000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-vrouter-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1734918823000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-12-23 01:03:26.567380", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7088-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 12/23/2024 01:53:46.284 7f8c9521d2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:53:46.285 7f8c9521d2e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1 build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-12-23 01:03:26.567380", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7088-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 2 num_core_per_socket = 2 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 12/23/2024 01:53:46.286 7f8c9521d2e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:53:46.286 7f8c9521d2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:53:46.286 7f8c9521d2e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 12/23/2024 01:53:46.286 7f8c9521d2e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:53:46.286 7f8c9521d2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:53:46.287 7f8c9521d2e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1 deleted = False process_info = [ << process_name = contrail-vrouter-agent process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1734918816000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-vrouter-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1734918823000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-12-23 01:03:26.567380", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7088-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 12/23/2024 01:53:46.287 7f8c9521d2e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:53:49.325 7f8c8f9d5748 [contrail-vrouter-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='an-jenkins-deploy-platform-juju-k8s-882-1', _node_type='Compute', _instance_id='0', _timestamp=1734918829325816, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-vrouter-agent', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1734918816000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-vrouter-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1734918823000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-12-23 01:03:26.567380", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7088-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 12/23/2024 01:53:49.326 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6800556, partition_space_available_1k=43803276, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=152444, cpu_share=0, mem_res=109044, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49288, cpu_share=0, mem_res=38400, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1242876, free=4867408, buffers=213604, cached=1615564, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.47, five_min_avg=0.28, fifteen_min_avg=0.2, cpu_share=0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 01:53:49.326 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:53:49.326 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1 deleted = False process_info = [ << process_name = contrail-vrouter-agent process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1734918816000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-vrouter-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1734918823000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-12-23 01:03:26.567380", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7088-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 12/23/2024 01:53:49.326 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:53:49.326 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:53:49.326 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6800556 partition_space_available_1k = 43803276 percentage_partition_space_used = 13 >> } all_core_file_list = [ ] process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 152444 cpu_share = 0 mem_res = 109044 >>contrail-vrouter-nodemgr : << mem_virt = 49288 cpu_share = 0 mem_res = 38400 >> } system_mem_usage = << total = 7939452 used = 1242876 free = 4867408 buffers = 213604 cached = 1615564 node_type = vrouter >> system_cpu_usage = << one_min_avg = 0.47 five_min_avg = 0.28 fifteen_min_avg = 0.2 cpu_share = 0 node_type = vrouter >> >> 12/23/2024 01:53:49.326 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/23/2024 01:53:50.062 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/23/2024 01:53:50.065 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Session Event: TCP Connected 12/23/2024 01:53:50.065 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/23/2024 01:53:50.065 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/23/2024 01:53:50.066 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = an-jenkins-deploy-platform-juju-k8s-882-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/23/2024 01:53:50.066 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/23/2024 01:53:50.066 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 12/23/2024 01:53:50.066 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 12/23/2024 01:53:50.067 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 12/23/2024 01:53:50.067 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 12/23/2024 01:53:50.067 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 12/23/2024 01:53:50.071 7f8c8d243a20 [contrail-vrouter-nodemgr] [DEBUG]: Extracted sandesh length: 0000001013 12/23/2024 01:53:50.071 7f8c8d243a20 [contrail-vrouter-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 12/23/2024 01:53:50.072 7f8c8f811d68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 12/23/2024 01:53:50.072 7f8c8f811d68 [contrail-vrouter-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 12/23/2024 01:53:50.072 7f8c8f811d68 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 12/23/2024 01:53:50.072 7f8c8f811d68 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 12/23/2024 01:53:50.072 7f8c8f811d68 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 12/23/2024 01:53:50.073 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 12/23/2024 01:53:50.074 7f8c8f5c22e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 12/23/2024 01:54:49.309 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6802052, partition_space_available_1k=43801780, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=154172, cpu_share=1.0, mem_res=110560, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49516, cpu_share=0.0, mem_res=38532, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1140744, free=4720136, buffers=213984, cached=1864588, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.8, five_min_avg=0.46, fifteen_min_avg=0.27, cpu_share=15.0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 01:55:49.319 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6857132, partition_space_available_1k=43746700, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=157900, cpu_share=1.0, mem_res=113820, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49468, cpu_share=0.0, mem_res=38416, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1160324, free=4633504, buffers=214184, cached=1931440, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.29, five_min_avg=0.38, fifteen_min_avg=0.25, cpu_share=1.0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 01:56:49.329 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6857172, partition_space_available_1k=43746660, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=159160, cpu_share=0.5, mem_res=115060, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49488, cpu_share=0.0, mem_res=38452, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1159964, free=4633732, buffers=214228, cached=1931528, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.14, five_min_avg=0.32, fifteen_min_avg=0.24, cpu_share=0.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 01:57:49.342 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6857224, partition_space_available_1k=43746608, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=160480, cpu_share=1.0, mem_res=116380, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49448, cpu_share=0.0, mem_res=38472, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1158136, free=4635456, buffers=214264, cached=1931596, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.6, five_min_avg=0.4, fifteen_min_avg=0.27, cpu_share=1.0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 01:58:08.157 7f8c8d225048 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 12/23/2024 01:58:49.368 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6857432, partition_space_available_1k=43746400, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=161788, cpu_share=0.5, mem_res=117568, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49844, cpu_share=0.0, mem_res=38760, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1162452, free=4626544, buffers=215292, cached=1935164, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.4, five_min_avg=0.37, fifteen_min_avg=0.27, cpu_share=4.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 01:59:49.366 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6857476, partition_space_available_1k=43746356, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=162168, cpu_share=0.0, mem_res=117832, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49856, cpu_share=0.0, mem_res=38448, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1195668, free=4583124, buffers=215760, cached=1944900, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.14, five_min_avg=0.3, fifteen_min_avg=0.25, cpu_share=1.0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 02:00:49.380 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7139400, partition_space_available_1k=43464432, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=174844, cpu_share=2.0, mem_res=122496, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49856, cpu_share=0.0, mem_res=38468, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1219904, free=4248728, buffers=220800, cached=2250020, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.55, five_min_avg=0.41, fifteen_min_avg=0.29, cpu_share=4.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 02:00:49.381 7f8c8f9d5748 [contrail-vrouter-nodemgr] [ERROR]: LB stats failure a bytes-like object is required, not 'str' 12/23/2024 02:01:49.387 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7141168, partition_space_available_1k=43462664, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=180824, cpu_share=2.5, mem_res=127984, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49864, cpu_share=0.0, mem_res=38452, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1269904, free=4189328, buffers=222452, cached=2257768, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.27, five_min_avg=0.36, fifteen_min_avg=0.28, cpu_share=3.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 02:01:49.388 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: Add uve <29bcf094-e8cc-441b-bfd1-622bf6a025a1, 1> in the [ObjectLBTable:UveLoadbalancerTrace] map 12/23/2024 02:02:49.396 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7141328, partition_space_available_1k=43462504, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=181864, cpu_share=1.0, mem_res=128716, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49884, cpu_share=0.0, mem_res=38492, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1283220, free=4175596, buffers=222472, cached=2258164, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.14, five_min_avg=0.31, fifteen_min_avg=0.27, cpu_share=1.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 02:03:29.639 7f8c8d225198 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 12/23/2024 02:03:49.416 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7141564, partition_space_available_1k=43462268, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=182920, cpu_share=0.5, mem_res=129640, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50036, cpu_share=0.0, mem_res=38760, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1287752, free=4169520, buffers=223224, cached=2258956, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.05, five_min_avg=0.25, fifteen_min_avg=0.25, cpu_share=1.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 02:04:49.418 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7141320, partition_space_available_1k=43462512, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=185148, cpu_share=0.5, mem_res=131488, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49880, cpu_share=0.0, mem_res=38468, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1249896, free=4209592, buffers=223272, cached=2256692, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.07, five_min_avg=0.22, fifteen_min_avg=0.24, cpu_share=3.0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 02:05:49.428 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7140672, partition_space_available_1k=43463160, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=186156, cpu_share=0.5, mem_res=132280, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49916, cpu_share=0.0, mem_res=38484, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1209316, free=4252560, buffers=223300, cached=2254276, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.1, five_min_avg=0.19, fifteen_min_avg=0.22, cpu_share=1.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 02:06:49.439 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7140728, partition_space_available_1k=43463104, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=186544, cpu_share=0.5, mem_res=132544, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49924, cpu_share=0.0, mem_res=38484, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1198420, free=4263676, buffers=223324, cached=2254032, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.03, five_min_avg=0.15, fifteen_min_avg=0.21, cpu_share=0.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 02:07:49.450 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7140776, partition_space_available_1k=43463056, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=186780, cpu_share=0.5, mem_res=132808, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49932, cpu_share=0.0, mem_res=38612, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1197560, free=4264464, buffers=223344, cached=2254084, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.19, five_min_avg=0.18, fifteen_min_avg=0.21, cpu_share=0.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 02:07:51.666 7f8c8d22d4e0 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 12/23/2024 02:08:49.467 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7473004, partition_space_available_1k=43130828, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=187644, cpu_share=1.0, mem_res=133696, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49884, cpu_share=0.0, mem_res=38728, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1204636, free=3898672, buffers=229652, cached=2606492, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.46, five_min_avg=0.27, fifteen_min_avg=0.24, cpu_share=4.0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 02:09:49.456 7f8c8f9d5748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='an-jenkins-deploy-platform-juju-k8s-882-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7472832, partition_space_available_1k=43131000, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=187912, cpu_share=0.5, mem_res=133712, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49876, cpu_share=0.0, mem_res=38488, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=7939452, used=1199076, free=3904864, buffers=229700, cached=2605812, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=0.22, five_min_avg=0.23, fifteen_min_avg=0.23, cpu_share=0.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 12/23/2024 02:10:28.313 7f8c8f3626d8 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map