10/21/2024 02:02:51.455 7fce10901f20 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 10/21/2024 02:02:51.455 7fce10901f20 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 10/21/2024 02:02:51.455 7fce10901f20 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 10/21/2024 02:02:51.468 7fce0a245248 [contrail-vrouter-nodemgr] [ERROR]: Cannot write http_port 8102 to /tmp/contrail-vrouter-nodemgr.0.http_port 10/21/2024 02:02:51.469 7fce0a245248 [contrail-vrouter-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8102 10/21/2024 02:02:51.470 7fce0a245448 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvStart] in state[none] 10/21/2024 02:02:51.471 7fce0a245448 [contrail-vrouter-nodemgr] [DEBUG]: Add uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:02:51.472 7fce0a245448 [contrail-vrouter-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/21/2024 02:02:51.472 7fce0a245448 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 10/21/2024 02:02:51.472 7fce0a245448 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:02:51.473 7fce0a245448 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/21/2024 02:02:51.473 7fce0a245448 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:02:51.473 7fce0a245448 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:02:51.473 7fce0a245448 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476171471926 collector_name = collector_ip = collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 10/21/2024 02:02:51.473 7fce0a245448 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:02:51.748 7fce10901f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'contrail-vrouter-agent', 'group': 'contrail-vrouter-agent', 'pid': 51049119556499157530388795323658209036370539944870534780791502201518117926938, 'start': '1729476167000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-vrouter-nodemgr', 'group': 'contrail-vrouter-nodemgr', 'pid': 4151547805593863287127740350341736764477668183312032593736534771017191908066, 'start': '1729476162000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 10/21/2024 02:02:51.931 7fce10901f20 [contrail-vrouter-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-439-1', _node_type='Compute', _instance_id='0', _timestamp=1729476171931347, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-vrouter-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 10/21/2024 02:02:51.932 7fce10901f20 [contrail-vrouter-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-439-1', _node_type='Compute', _instance_id='0', _timestamp=1729476171932064, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-vrouter-agent', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476167000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-vrouter-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476162000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 10/21/2024 02:02:51.936 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:02:51.937 7fce0a245848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 10/21/2024 02:02:51.937 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:02:51.937 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:02:51.937 7fce0a245848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/21/2024 02:02:51.937 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:02:51.938 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:02:51.938 7fce0a245848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 deleted = False process_info = [ << process_name = contrail-vrouter-agent process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476167000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-vrouter-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476162000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 10/21/2024 02:02:51.938 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:02:55.271 7fce10901f20 [contrail-vrouter-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-439-1', _node_type='Compute', _instance_id='0', _timestamp=1729476175270814, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-vrouter-agent', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476167000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-vrouter-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476162000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 10/21/2024 02:02:55.271 7fce10901f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51425600, partition_space_available_1k=29664884, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=220208, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=65976, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14564980, free=11775764, buffers=619404, cached=5704968, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=12.16, five_min_avg=7.85, fifteen_min_avg=4.42, cpu_share=0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:02:55.272 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:02:55.272 7fce0a245848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 deleted = False process_info = [ << process_name = contrail-vrouter-agent process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476167000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-vrouter-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476162000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 10/21/2024 02:02:55.272 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:02:55.272 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:02:55.272 7fce0a245848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51425600 partition_space_available_1k = 29664884 percentage_partition_space_used = 63 >> } all_core_file_list = [ ] process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 220208 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 65976 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14564980 free = 11775764 buffers = 619404 cached = 5704968 node_type = vrouter >> system_cpu_usage = << one_min_avg = 12.16 five_min_avg = 7.85 fifteen_min_avg = 4.42 cpu_share = 0 node_type = vrouter >> >> 10/21/2024 02:02:55.272 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:02:55.471 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:03:00.477 7fce0a245848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:03:00.477 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:03:00.477 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:03:00.477 7fce0a245848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:03:00.477 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:03:00.478 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:03:00.478 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:03:00.478 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:00.479 7fce0a245848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:03:00.479 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:00.479 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:00.479 7fce0a245848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476171471926 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:03:00.479 7fce0a245848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:05.891 7f4b7a965f20 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 10/21/2024 02:03:05.891 7f4b7a965f20 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 10/21/2024 02:03:05.892 7f4b7a965f20 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 10/21/2024 02:03:05.902 7f4b74273248 [contrail-vrouter-nodemgr] [ERROR]: Cannot write http_port 8102 to /tmp/contrail-vrouter-nodemgr.0.http_port 10/21/2024 02:03:05.902 7f4b74273248 [contrail-vrouter-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8102 10/21/2024 02:03:05.903 7f4b74273448 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvStart] in state[none] 10/21/2024 02:03:05.904 7f4b74273448 [contrail-vrouter-nodemgr] [DEBUG]: Add uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:03:05.905 7f4b74273448 [contrail-vrouter-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/21/2024 02:03:05.906 7f4b74273448 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 10/21/2024 02:03:05.906 7f4b74273448 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:05.906 7f4b74273448 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/21/2024 02:03:05.906 7f4b74273448 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:05.906 7f4b74273448 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:05.906 7f4b74273448 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 10/21/2024 02:03:05.906 7f4b74273448 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:06.092 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'contrail-vrouter-agent', 'group': 'contrail-vrouter-agent', 'pid': 70923211800223489800394440517861667434760872825901175744718470971095164798056, 'start': '1729476177000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-vrouter-nodemgr', 'group': 'contrail-vrouter-nodemgr', 'pid': 107285678254279989609491158055290942347999191790799538730493514655640764596232, 'start': '1729476184000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 10/21/2024 02:03:06.250 7f4b7a965f20 [contrail-vrouter-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-439-1', _node_type='Compute', _instance_id='0', _timestamp=1729476186249807, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-vrouter-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 10/21/2024 02:03:06.250 7f4b7a965f20 [contrail-vrouter-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-439-1', _node_type='Compute', _instance_id='0', _timestamp=1729476186250427, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-vrouter-agent', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476177000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-vrouter-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476184000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 10/21/2024 02:03:06.254 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:06.255 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 10/21/2024 02:03:06.255 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:06.255 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:06.255 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/21/2024 02:03:06.255 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:06.256 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:06.256 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 deleted = False process_info = [ << process_name = contrail-vrouter-agent process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476177000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-vrouter-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476184000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 10/21/2024 02:03:06.256 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:09.281 7f4b7a965f20 [contrail-vrouter-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-439-1', _node_type='Compute', _instance_id='0', _timestamp=1729476189281325, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-vrouter-agent', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476177000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-vrouter-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1729476184000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 10/21/2024 02:03:09.282 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51433512, partition_space_available_1k=29656972, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=143524, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53000, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14795052, free=11516312, buffers=621028, cached=5732724, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=10.31, five_min_avg=7.66, fifteen_min_avg=4.42, cpu_share=0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:03:09.282 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:09.282 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 deleted = False process_info = [ << process_name = contrail-vrouter-agent process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476177000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-vrouter-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1729476184000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-21 01:11:05.701862", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6068-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 10/21/2024 02:03:09.282 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:09.282 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:09.283 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51433512 partition_space_available_1k = 29656972 percentage_partition_space_used = 63 >> } all_core_file_list = [ ] process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 143524 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53000 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14795052 free = 11516312 buffers = 621028 cached = 5732724 node_type = vrouter >> system_cpu_usage = << one_min_avg = 10.31 five_min_avg = 7.66 fifteen_min_avg = 4.42 cpu_share = 0 node_type = vrouter >> >> 10/21/2024 02:03:09.283 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:09.905 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:03:10.997 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:03:14.910 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:03:14.910 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:03:14.910 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:03:14.910 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:03:14.910 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:03:14.910 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:03:14.911 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:03:14.911 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:14.911 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:03:14.911 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:14.912 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:14.912 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:03:14.912 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:18.912 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:03:23.916 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:03:23.916 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:03:23.917 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:03:23.917 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:03:23.917 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:03:23.917 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:03:23.918 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:03:23.918 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:23.918 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:03:23.918 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:23.918 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:23.918 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:03:23.918 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:25.149 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:03:27.919 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:03:32.923 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:03:32.924 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:03:32.924 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:03:32.924 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:03:32.924 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:03:32.924 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:03:32.925 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:03:32.925 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:32.925 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:03:32.925 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:32.925 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:32.925 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:03:32.925 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:36.924 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:03:41.929 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:03:41.929 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:03:41.929 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:03:41.929 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:03:41.929 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:03:41.930 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:03:41.930 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:03:41.930 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:41.931 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:03:41.931 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:41.931 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:41.931 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:03:41.931 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:45.930 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:03:50.936 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:03:50.936 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:03:50.936 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:03:50.936 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:03:50.936 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:03:50.936 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:03:50.937 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:03:50.937 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:50.937 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:03:50.937 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:50.937 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:50.937 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:03:50.937 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:54.937 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:03:59.941 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:03:59.942 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:03:59.942 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:03:59.942 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:03:59.942 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:03:59.942 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:03:59.942 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:03:59.943 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:59.943 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:03:59.943 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:59.943 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:03:59.943 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:03:59.943 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:03.943 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:04:08.946 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:04:08.946 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:04:08.946 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:04:08.947 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:04:08.947 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:04:08.947 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:04:08.947 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:04:08.948 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:08.948 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:04:08.948 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:08.948 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:08.948 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:04:08.948 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:09.286 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51430976, partition_space_available_1k=29659508, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=145400, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53284, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14607664, free=11674276, buffers=622876, cached=5760300, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=5.26, five_min_avg=6.7, fifteen_min_avg=4.29, cpu_share=3.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:04:09.286 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:09.286 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51430976 partition_space_available_1k = 29659508 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 145400 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53284 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14607664 free = 11674276 buffers = 622876 cached = 5760300 node_type = vrouter >> system_cpu_usage = << one_min_avg = 5.26 five_min_avg = 6.7 fifteen_min_avg = 4.29 cpu_share = 3.5 node_type = vrouter >> >> 10/21/2024 02:04:09.286 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:12.947 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:04:17.952 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:04:17.952 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:04:17.952 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:04:17.953 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:04:17.953 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:04:17.953 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:04:17.953 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:04:17.953 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:17.953 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:04:17.953 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:17.954 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:17.954 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:04:17.954 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:21.954 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:04:26.959 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:04:26.959 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:04:26.959 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:04:26.960 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:04:26.960 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:04:26.960 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:04:26.960 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:04:26.960 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:26.960 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:04:26.961 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:26.961 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:26.961 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:04:26.961 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:30.961 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:04:35.965 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:04:35.965 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:04:35.965 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:04:35.966 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:04:35.966 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:04:35.966 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:04:35.966 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:04:35.966 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:35.966 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:04:35.966 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:35.967 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:35.967 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:04:35.967 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:39.966 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:04:44.198 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:04:44.968 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:04:44.968 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:04:44.968 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:04:44.969 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:04:44.969 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:04:44.969 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:04:44.969 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:04:44.969 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:44.970 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:04:44.970 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:44.970 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:44.970 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:04:44.970 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:48.969 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:04:53.972 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:04:53.972 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:04:53.972 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:04:53.972 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:04:53.972 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:04:53.973 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:04:53.973 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:04:53.973 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:53.973 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:04:53.973 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:53.973 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:53.974 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:04:53.974 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:04:57.973 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:04:59.453 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:05:02.977 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:05:02.977 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:05:02.977 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:02.977 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:05:02.978 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:02.978 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:05:02.978 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:05:02.978 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:02.979 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:05:02.979 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:02.979 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:02.979 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:05:02.979 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:06.978 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:05:09.287 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51438528, partition_space_available_1k=29651956, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=146580, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53372, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14692872, free=11509868, buffers=626044, cached=5836332, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=5.19, five_min_avg=6.43, fifteen_min_avg=4.35, cpu_share=2.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:05:11.982 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:05:11.982 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:05:11.983 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:11.983 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:05:11.983 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:11.983 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:11.983 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51438528 partition_space_available_1k = 29651956 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 146580 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53372 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14692872 free = 11509868 buffers = 626044 cached = 5836332 node_type = vrouter >> system_cpu_usage = << one_min_avg = 5.19 five_min_avg = 6.43 fifteen_min_avg = 4.35 cpu_share = 2.5 node_type = vrouter >> >> 10/21/2024 02:05:11.983 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:11.983 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:05:11.984 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:05:11.984 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:11.984 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:05:11.984 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:11.984 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:11.984 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:05:11.984 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:13.540 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:05:15.983 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:05:20.988 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:05:20.988 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:05:20.988 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:20.988 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:05:20.989 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:20.989 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:05:20.989 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:05:20.989 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:20.989 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:05:20.989 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:20.989 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:20.990 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:05:20.990 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:24.989 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:05:29.993 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:05:29.993 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:05:29.993 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:29.993 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:05:29.993 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:29.993 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:05:29.994 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:05:29.994 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:29.994 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:05:29.994 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:29.994 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:29.994 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:05:29.994 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:33.995 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:05:38.997 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:05:38.997 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:05:38.997 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:38.997 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:05:38.998 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:38.998 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:05:38.998 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:05:38.998 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:38.998 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:05:38.998 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:38.999 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:38.999 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:05:38.999 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:42.998 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:05:48.003 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:05:48.004 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:05:48.004 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:48.004 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:05:48.004 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:48.004 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:05:48.005 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:05:48.005 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:48.005 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:05:48.005 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:48.005 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:48.005 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:05:48.005 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:52.006 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:05:53.461 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:05:57.010 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:05:57.011 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:05:57.011 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:57.011 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:05:57.011 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:05:57.011 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:05:57.012 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:05:57.012 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:57.012 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:05:57.012 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:57.012 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:05:57.012 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:05:57.012 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:01.014 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:06:06.024 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:06:06.024 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:06:06.025 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:06:06.025 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:06:06.025 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:06:06.025 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:06:06.025 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:06:06.025 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:06.026 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:06:06.026 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:06.026 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:06.026 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:06:06.026 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:09.310 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51441948, partition_space_available_1k=29648536, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=147624, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53392, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14957752, free=10189660, buffers=676976, cached=6840728, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=8.0, five_min_avg=6.66, fifteen_min_avg=4.54, cpu_share=4.12, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:06:09.318 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:09.320 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51441948 partition_space_available_1k = 29648536 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 147624 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53392 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14957752 free = 10189660 buffers = 676976 cached = 6840728 node_type = vrouter >> system_cpu_usage = << one_min_avg = 8.0 five_min_avg = 6.66 fifteen_min_avg = 4.54 cpu_share = 4.12 node_type = vrouter >> >> 10/21/2024 02:06:09.320 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:10.025 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:06:15.030 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:06:15.030 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:06:15.030 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:06:15.030 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:06:15.030 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:06:15.030 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:06:15.031 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:06:15.031 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:15.031 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:06:15.031 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:15.031 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:15.031 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:06:15.031 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:19.033 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:06:24.035 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:06:24.035 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:06:24.035 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:06:24.035 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:06:24.035 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:06:24.035 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:06:24.036 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:06:24.036 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:24.036 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:06:24.036 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:24.036 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:24.036 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:06:24.036 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:28.037 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:06:33.043 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:06:33.043 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:06:33.043 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:06:33.043 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:06:33.043 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:06:33.044 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:06:33.044 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:06:33.044 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:33.044 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:06:33.044 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:33.044 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:33.044 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:06:33.044 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:37.045 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:06:42.048 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:06:42.049 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:06:42.049 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:06:42.049 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:06:42.049 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:06:42.049 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:06:42.049 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:06:42.050 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:42.050 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:06:42.050 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:42.050 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:42.050 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:06:42.050 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:46.051 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:06:51.057 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:06:51.057 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:06:51.057 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:06:51.057 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:06:51.057 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:06:51.057 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:06:51.058 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:06:51.058 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:51.058 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:06:51.058 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:51.058 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:51.058 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:06:51.058 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:06:55.057 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:07:00.063 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:07:00.063 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:07:00.063 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:07:00.063 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:07:00.063 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:07:00.063 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:07:00.064 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:07:00.064 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:00.064 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:07:00.064 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:00.064 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:00.064 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:07:00.064 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:04.063 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:07:09.066 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:07:09.067 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:07:09.067 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:07:09.068 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:07:09.068 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:07:09.068 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:07:09.069 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:07:09.069 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:09.069 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:07:09.070 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:09.071 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:09.071 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:07:09.071 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:09.288 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51451404, partition_space_available_1k=29639080, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=148692, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53460, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14646624, free=10238272, buffers=678364, cached=7101856, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=5.86, five_min_avg=6.46, fifteen_min_avg=4.61, cpu_share=4.0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:07:09.289 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:09.289 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51451404 partition_space_available_1k = 29639080 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 148692 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53460 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14646624 free = 10238272 buffers = 678364 cached = 7101856 node_type = vrouter >> system_cpu_usage = << one_min_avg = 5.86 five_min_avg = 6.46 fifteen_min_avg = 4.61 cpu_share = 4.0 node_type = vrouter >> >> 10/21/2024 02:07:09.289 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:13.069 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:07:18.074 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:07:18.075 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:07:18.075 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:07:18.075 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:07:18.075 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:07:18.075 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:07:18.076 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:07:18.076 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:18.076 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:07:18.076 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:18.076 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:18.076 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:07:18.076 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:22.077 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:07:27.081 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:07:27.081 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:07:27.081 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:07:27.081 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:07:27.081 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:07:27.082 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:07:27.082 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:07:27.082 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:27.082 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:07:27.082 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:27.082 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:27.083 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:07:27.083 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:31.083 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:07:36.089 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:07:36.089 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:07:36.089 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:07:36.089 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:07:36.089 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:07:36.089 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:07:36.090 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:07:36.090 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:36.090 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:07:36.091 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:36.091 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:36.091 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:07:36.091 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:40.089 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:07:45.093 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:07:45.093 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:07:45.093 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:07:45.093 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:07:45.093 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:07:45.094 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:07:45.094 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:07:45.094 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:45.094 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:07:45.094 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:45.094 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:45.095 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:07:45.095 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:49.093 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:07:54.097 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:07:54.097 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:07:54.097 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:07:54.098 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:07:54.098 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:07:54.098 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:07:54.099 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:07:54.099 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:54.099 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:07:54.099 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:54.099 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:54.099 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:07:54.099 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:07:58.099 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:08:03.104 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:08:03.104 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:08:03.104 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:03.104 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:08:03.104 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:03.104 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:08:03.105 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:08:03.105 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:03.105 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:08:03.105 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:03.105 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:03.105 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:08:03.105 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:07.104 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:08:09.300 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51455192, partition_space_available_1k=29635292, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=149580, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53464, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14658028, free=10221224, buffers=678364, cached=7107500, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=3.65, five_min_avg=5.67, fifteen_min_avg=4.45, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:08:12.108 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:08:12.109 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:08:12.109 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:12.109 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:08:12.109 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:12.109 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:12.110 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51455192 partition_space_available_1k = 29635292 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 149580 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53464 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14658028 free = 10221224 buffers = 678364 cached = 7107500 node_type = vrouter >> system_cpu_usage = << one_min_avg = 3.65 five_min_avg = 5.67 fifteen_min_avg = 4.45 cpu_share = 2.62 node_type = vrouter >> >> 10/21/2024 02:08:12.110 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:12.110 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:08:12.111 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:08:12.111 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:12.111 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:08:12.111 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:12.111 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:12.111 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:08:12.112 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:16.112 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:08:21.117 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:08:21.117 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:08:21.117 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:21.117 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:08:21.118 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:21.118 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:08:21.118 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:08:21.118 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:21.118 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:08:21.119 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:21.119 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:21.119 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:08:21.119 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:25.122 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:08:30.129 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:08:30.129 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:08:30.130 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:30.130 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:08:30.130 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:30.130 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:08:30.130 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:08:30.130 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:30.131 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:08:30.131 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:30.131 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:30.131 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:08:30.131 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:34.132 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:08:39.135 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:08:39.135 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:08:39.135 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:39.136 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:08:39.136 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:39.136 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:08:39.136 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:08:39.136 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:39.136 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:08:39.136 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:39.137 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:39.137 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:08:39.137 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:43.137 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:08:48.141 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:08:48.142 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:08:48.142 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:48.142 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:08:48.142 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:48.142 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:08:48.142 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:08:48.143 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:48.143 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:08:48.143 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:48.143 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:48.143 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:08:48.143 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:52.142 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:08:57.144 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:08:57.144 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:08:57.144 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:57.144 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:08:57.144 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:08:57.144 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:08:57.145 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:08:57.145 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:57.145 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:08:57.145 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:57.145 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:57.145 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:08:57.145 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:08:58.594 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:09:01.145 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:09:06.149 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:09:06.149 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:09:06.149 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:09:06.149 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:09:06.149 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:09:06.150 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:09:06.150 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:09:06.150 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:06.150 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:09:06.150 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:06.150 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:06.151 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:09:06.151 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:09.294 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51462920, partition_space_available_1k=29627564, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=149800, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53492, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14714212, free=10155456, buffers=678460, cached=7116988, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.49, five_min_avg=4.97, fifteen_min_avg=4.29, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:09:09.294 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:09.295 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51462920 partition_space_available_1k = 29627564 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 149800 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53492 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14714212 free = 10155456 buffers = 678460 cached = 7116988 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.49 five_min_avg = 4.97 fifteen_min_avg = 4.29 cpu_share = 2.62 node_type = vrouter >> >> 10/21/2024 02:09:09.295 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:10.150 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:09:12.492 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:09:15.153 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:09:15.153 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:09:15.153 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:09:15.153 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:09:15.153 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:09:15.153 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:09:15.154 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:09:15.154 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:15.154 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:09:15.154 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:15.154 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:15.155 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:09:15.155 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:19.155 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:09:24.161 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:09:24.161 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:09:24.161 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:09:24.161 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:09:24.162 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:09:24.162 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:09:24.162 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:09:24.162 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:24.162 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:09:24.162 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:24.163 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:24.163 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:09:24.163 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:28.161 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:09:29.489 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:09:33.166 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:09:33.166 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:09:33.166 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:09:33.166 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:09:33.166 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:09:33.166 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:09:33.167 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:09:33.167 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:33.167 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:09:33.167 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:33.167 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:33.167 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:09:33.168 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:37.169 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:09:42.171 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:09:42.171 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:09:42.171 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:09:42.171 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:09:42.171 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:09:42.171 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:09:42.172 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:09:42.172 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:42.172 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:09:42.172 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:42.172 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:42.173 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:09:42.173 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:46.173 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:09:51.179 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:09:51.180 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:09:51.180 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:09:51.180 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:09:51.180 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:09:51.180 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:09:51.180 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:09:51.181 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:51.181 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:09:51.181 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:51.181 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:51.181 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:09:51.181 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:09:54.305 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:09:55.180 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:10:00.184 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:10:00.184 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:10:00.184 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:10:00.185 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:10:00.185 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:10:00.185 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:10:00.185 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:10:00.185 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:00.186 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:10:00.186 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:00.186 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:00.186 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:10:00.186 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:04.187 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:10:09.189 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:10:09.190 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:10:09.190 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:10:09.190 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:10:09.190 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:10:09.191 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:10:09.192 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:10:09.192 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:09.192 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:10:09.192 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:09.192 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:09.193 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:10:09.193 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:09.304 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51463080, partition_space_available_1k=29627404, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=150084, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53560, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14683024, free=10184280, buffers=678464, cached=7119348, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.91, five_min_avg=4.6, fifteen_min_avg=4.2, cpu_share=2.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:10:09.304 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:09.305 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51463080 partition_space_available_1k = 29627404 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 150084 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53560 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14683024 free = 10184280 buffers = 678464 cached = 7119348 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.91 five_min_avg = 4.6 fifteen_min_avg = 4.2 cpu_share = 2.75 node_type = vrouter >> >> 10/21/2024 02:10:09.305 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:13.190 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:10:18.196 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:10:18.196 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:10:18.196 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:10:18.196 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:10:18.196 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:10:18.196 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:10:18.197 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:10:18.197 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:18.197 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:10:18.197 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:18.197 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:18.197 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:10:18.197 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:22.198 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:10:27.200 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:10:27.201 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:10:27.201 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:10:27.201 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:10:27.201 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:10:27.201 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:10:27.202 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:10:27.202 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:27.202 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:10:27.202 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:27.202 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:27.202 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:10:27.202 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:31.202 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:10:36.207 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:10:36.208 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:10:36.208 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:10:36.208 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:10:36.208 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:10:36.208 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:10:36.209 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:10:36.209 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:36.209 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:10:36.209 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:36.209 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:36.209 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:10:36.209 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:40.209 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:10:45.214 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:10:45.215 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:10:45.215 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:10:45.215 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:10:45.215 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:10:45.215 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:10:45.216 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:10:45.216 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:45.216 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:10:45.216 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:45.216 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:45.216 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:10:45.217 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:49.217 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:10:54.223 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:10:54.223 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:10:54.223 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:10:54.224 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:10:54.224 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:10:54.224 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:10:54.224 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:10:54.224 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:54.225 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:10:54.225 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:54.225 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:54.225 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:10:54.225 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:10:58.224 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:11:03.229 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:11:03.229 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:11:03.229 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:03.229 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:11:03.229 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:03.229 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:11:03.230 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:11:03.230 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:03.230 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:11:03.230 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:03.230 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:03.230 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:11:03.230 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:07.229 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:11:09.298 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51466836, partition_space_available_1k=29623648, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=150208, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53592, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14642928, free=10219744, buffers=678476, cached=7123968, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.34, five_min_avg=4.15, fifteen_min_avg=4.07, cpu_share=2.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:11:12.231 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:11:12.231 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:11:12.231 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:12.231 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:11:12.232 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:12.232 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:12.232 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51466836 partition_space_available_1k = 29623648 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 150208 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53592 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14642928 free = 10219744 buffers = 678476 cached = 7123968 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.34 five_min_avg = 4.15 fifteen_min_avg = 4.07 cpu_share = 2.75 node_type = vrouter >> >> 10/21/2024 02:11:12.232 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:12.232 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:11:12.232 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:11:12.233 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:12.233 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:11:12.233 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:12.233 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:12.233 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:11:12.233 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:16.234 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:11:21.239 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:11:21.239 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:11:21.240 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:21.240 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:11:21.240 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:21.240 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:11:21.240 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:11:21.240 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:21.241 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:11:21.241 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:21.241 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:21.241 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:11:21.241 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:25.242 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:11:30.245 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:11:30.245 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:11:30.245 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:30.245 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:11:30.245 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:30.245 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:11:30.246 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:11:30.246 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:30.246 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:11:30.246 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:30.246 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:30.246 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:11:30.246 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:34.245 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:11:39.251 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:11:39.251 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:11:39.251 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:39.251 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:11:39.251 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:39.251 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:11:39.252 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:11:39.252 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:39.252 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:11:39.252 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:39.252 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:39.253 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:11:39.253 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:43.251 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:11:48.256 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:11:48.256 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:11:48.256 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:48.256 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:11:48.256 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:48.257 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:11:48.257 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:11:48.257 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:48.257 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:11:48.257 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:48.257 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:48.257 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:11:48.257 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:52.260 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:11:57.265 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:11:57.265 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:11:57.265 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:57.265 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:11:57.266 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:11:57.266 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:11:57.266 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:11:57.266 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:57.266 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:11:57.266 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:57.266 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:11:57.266 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:11:57.267 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:01.266 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:12:06.272 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:12:06.273 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:12:06.273 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:12:06.273 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:12:06.274 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:12:06.274 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:12:06.275 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:12:06.275 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:06.275 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:12:06.275 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:06.275 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:06.275 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:12:06.275 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:09.298 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51477052, partition_space_available_1k=29613432, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=150388, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53644, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14711664, free=10129400, buffers=678476, cached=7145576, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=7.95, five_min_avg=5.39, fifteen_min_avg=4.5, cpu_share=4.88, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:12:09.299 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:09.299 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51477052 partition_space_available_1k = 29613432 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 150388 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53644 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14711664 free = 10129400 buffers = 678476 cached = 7145576 node_type = vrouter >> system_cpu_usage = << one_min_avg = 7.95 five_min_avg = 5.39 fifteen_min_avg = 4.5 cpu_share = 4.88 node_type = vrouter >> >> 10/21/2024 02:12:09.299 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:10.272 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:12:15.276 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:12:15.276 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:12:15.276 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:12:15.276 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:12:15.277 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:12:15.277 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:12:15.277 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:12:15.278 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:15.278 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:12:15.278 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:15.278 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:15.278 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:12:15.278 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:19.278 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:12:24.284 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:12:24.284 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:12:24.284 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:12:24.285 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:12:24.285 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:12:24.285 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:12:24.285 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:12:24.285 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:24.285 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:12:24.286 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:24.286 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:24.286 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:12:24.286 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:28.284 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:12:33.289 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:12:33.289 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:12:33.289 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:12:33.290 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:12:33.290 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:12:33.290 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:12:33.290 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:12:33.291 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:33.291 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:12:33.291 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:33.291 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:33.291 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:12:33.291 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:37.292 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:12:42.297 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:12:42.297 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:12:42.297 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:12:42.297 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:12:42.297 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:12:42.297 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:12:42.298 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:12:42.298 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:42.298 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:12:42.298 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:42.298 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:42.298 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:12:42.298 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:46.299 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:12:51.303 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:12:51.304 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:12:51.304 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:12:51.304 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:12:51.304 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:12:51.304 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:12:51.305 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:12:51.305 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:51.305 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:12:51.305 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:51.305 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:51.305 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:12:51.305 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:12:55.307 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:13:00.309 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:13:00.310 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:13:00.310 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:00.310 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:13:00.310 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:00.310 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:13:00.311 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:13:00.311 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:00.311 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:13:00.311 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:00.311 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:00.311 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:13:00.311 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:04.313 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:13:09.316 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51482124, partition_space_available_1k=29608360, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=150508, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53680, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14771640, free=10059528, buffers=678484, cached=7155464, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=4.48, five_min_avg=4.85, fifteen_min_avg=4.36, cpu_share=2.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:13:09.323 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:13:09.323 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:13:09.323 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:09.324 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:13:09.324 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:09.324 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:09.324 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51482124 partition_space_available_1k = 29608360 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 150508 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53680 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14771640 free = 10059528 buffers = 678484 cached = 7155464 node_type = vrouter >> system_cpu_usage = << one_min_avg = 4.48 five_min_avg = 4.85 fifteen_min_avg = 4.36 cpu_share = 2.75 node_type = vrouter >> >> 10/21/2024 02:13:09.324 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:09.324 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:13:09.325 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:13:09.325 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:09.325 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:13:09.325 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:09.326 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:09.326 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:13:09.326 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:09.494 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:13:13.326 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:13:18.330 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:13:18.330 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:13:18.330 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:18.330 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:13:18.330 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:18.330 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:13:18.331 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:13:18.331 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:18.331 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:13:18.331 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:18.331 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:18.331 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:13:18.331 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:22.333 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:13:23.307 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:13:27.337 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:13:27.337 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:13:27.337 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:27.337 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:13:27.338 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:27.338 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:13:27.338 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:13:27.338 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:27.338 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:13:27.338 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:27.339 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:27.339 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:13:27.339 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:31.339 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:13:36.344 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:13:36.344 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:13:36.344 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:36.345 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:13:36.345 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:36.345 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:13:36.345 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:13:36.346 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:36.346 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:13:36.346 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:36.346 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:36.346 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:13:36.346 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:40.345 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:13:40.461 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:13:45.346 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:13:45.347 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:13:45.347 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:45.347 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:13:45.347 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:45.347 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:13:45.347 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:13:45.348 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:45.348 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:13:45.348 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:45.348 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:45.348 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:13:45.348 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:49.349 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:13:54.354 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:13:54.354 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:13:54.354 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:54.355 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:13:54.355 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:13:54.355 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:13:54.355 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:13:54.355 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:54.355 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:13:54.356 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:54.356 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:54.356 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:13:54.356 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:13:58.357 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:14:03.363 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:14:03.363 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:14:03.363 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:03.363 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:14:03.363 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:03.364 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:14:03.364 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:14:03.364 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:03.364 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:14:03.364 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:03.364 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:03.365 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:14:03.365 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:07.363 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:14:09.301 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51486128, partition_space_available_1k=29604356, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=150720, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53712, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14757784, free=10068812, buffers=678492, cached=7160028, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=3.07, five_min_avg=4.4, fifteen_min_avg=4.24, cpu_share=2.88, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:14:12.367 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:14:12.368 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:14:12.368 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:12.368 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:14:12.368 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:12.368 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:12.368 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51486128 partition_space_available_1k = 29604356 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 150720 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53712 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14757784 free = 10068812 buffers = 678492 cached = 7160028 node_type = vrouter >> system_cpu_usage = << one_min_avg = 3.07 five_min_avg = 4.4 fifteen_min_avg = 4.24 cpu_share = 2.88 node_type = vrouter >> >> 10/21/2024 02:14:12.368 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:12.369 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:14:12.369 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:14:12.369 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:12.369 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:14:12.370 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:12.370 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:12.370 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:14:12.370 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:16.369 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:14:21.373 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:14:21.373 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:14:21.373 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:21.373 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:14:21.374 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:21.374 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:14:21.374 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:14:21.374 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:21.374 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:14:21.374 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:21.374 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:21.375 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:14:21.375 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:25.375 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:14:30.377 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:14:30.377 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:14:30.377 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:30.377 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:14:30.378 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:30.378 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:14:30.378 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:14:30.378 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:30.379 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:14:30.379 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:30.379 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:30.379 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:14:30.379 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:34.380 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:14:39.386 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:14:39.386 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:14:39.386 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:39.387 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:14:39.387 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:39.387 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:14:39.388 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:14:39.388 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:39.388 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:14:39.388 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:39.388 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:39.388 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:14:39.389 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:43.389 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:14:48.394 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:14:48.394 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:14:48.394 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:48.395 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:14:48.395 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:48.395 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:14:48.395 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:14:48.395 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:48.395 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:14:48.396 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:48.396 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:48.396 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:14:48.396 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:52.396 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:14:57.401 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:14:57.401 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:14:57.401 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:57.401 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:14:57.401 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:14:57.402 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:14:57.402 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:14:57.402 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:57.402 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:14:57.402 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:57.402 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:14:57.402 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:14:57.403 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:01.402 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:15:06.403 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:15:06.404 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:15:06.404 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:15:06.404 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:15:06.404 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:15:06.404 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:15:06.405 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:15:06.405 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:06.405 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:15:06.405 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:06.406 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:06.406 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:15:06.406 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:09.303 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51490072, partition_space_available_1k=29600412, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=150808, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53752, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14789552, free=10031272, buffers=678528, cached=7165764, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.35, five_min_avg=3.95, fifteen_min_avg=4.09, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:15:09.303 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:09.303 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51490072 partition_space_available_1k = 29600412 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 150808 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53752 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14789552 free = 10031272 buffers = 678528 cached = 7165764 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.35 five_min_avg = 3.95 fifteen_min_avg = 4.09 cpu_share = 2.62 node_type = vrouter >> >> 10/21/2024 02:15:09.303 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:10.404 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:15:15.406 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:15:15.407 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:15:15.407 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:15:15.407 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:15:15.407 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:15:15.407 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:15:15.408 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:15:15.408 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:15.408 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:15:15.408 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:15.408 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:15.408 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:15:15.408 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:19.409 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:15:24.415 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:15:24.415 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:15:24.415 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:15:24.415 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:15:24.415 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:15:24.415 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:15:24.416 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:15:24.416 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:24.416 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:15:24.416 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:24.416 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:24.416 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:15:24.416 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:26.453 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:15:28.415 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:15:33.418 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:15:33.418 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:15:33.418 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:15:33.418 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:15:33.418 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:15:33.418 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:15:33.419 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:15:33.419 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:33.419 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:15:33.419 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:33.419 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:33.420 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:15:33.420 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:37.421 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:15:42.426 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:15:42.426 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:15:42.426 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:15:42.426 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:15:42.426 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:15:42.426 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:15:42.427 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:15:42.427 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:42.427 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:15:42.427 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:42.427 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:42.427 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:15:42.427 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:46.426 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:15:51.431 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:15:51.431 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:15:51.431 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:15:51.431 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:15:51.431 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:15:51.431 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:15:51.432 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:15:51.432 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:51.432 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:15:51.432 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:51.432 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:51.432 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:15:51.432 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:15:55.433 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:16:00.436 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:16:00.436 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:16:00.436 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:00.436 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:16:00.436 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:00.437 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:16:00.437 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:16:00.437 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:00.437 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:16:00.437 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:00.437 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:00.437 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:16:00.438 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:04.439 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:16:09.311 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51493700, partition_space_available_1k=29596784, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=150924, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53812, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14798348, free=10016452, buffers=678536, cached=7171780, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.1, five_min_avg=3.56, fifteen_min_avg=3.95, cpu_share=2.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:16:09.441 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:16:09.441 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:16:09.441 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:09.441 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:16:09.441 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:09.441 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:09.442 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51493700 partition_space_available_1k = 29596784 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 150924 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53812 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14798348 free = 10016452 buffers = 678536 cached = 7171780 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.1 five_min_avg = 3.56 fifteen_min_avg = 3.95 cpu_share = 2.75 node_type = vrouter >> >> 10/21/2024 02:16:09.442 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:09.442 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:16:09.442 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:16:09.442 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:09.443 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:16:09.443 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:09.443 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:09.443 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:16:09.443 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:13.444 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:16:18.448 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:16:18.448 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:16:18.448 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:18.449 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:16:18.449 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:18.449 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:16:18.449 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:16:18.449 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:18.450 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:16:18.450 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:18.450 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:18.450 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:16:18.450 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:22.449 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:16:27.454 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:16:27.454 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:16:27.454 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:27.455 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:16:27.455 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:27.455 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:16:27.455 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:16:27.455 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:27.456 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:16:27.456 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:27.456 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:27.456 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:16:27.456 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:31.455 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:16:36.460 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:16:36.460 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:16:36.460 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:36.460 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:16:36.461 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:36.461 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:16:36.461 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:16:36.461 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:36.461 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:16:36.461 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:36.461 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:36.462 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:16:36.462 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:40.461 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:16:45.466 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:16:45.467 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:16:45.467 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:45.467 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:16:45.467 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:45.467 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:16:45.467 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:16:45.468 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:45.468 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:16:45.468 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:45.468 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:45.468 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:16:45.468 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:49.469 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:16:54.475 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:16:54.475 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:16:54.475 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:54.476 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:16:54.476 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:16:54.476 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:16:54.477 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:16:54.477 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:54.477 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:16:54.477 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:54.477 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:54.477 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:16:54.478 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:16:58.477 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:17:03.481 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:17:03.481 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:17:03.481 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:03.481 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:17:03.481 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:03.482 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:17:03.482 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:17:03.482 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:03.482 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:17:03.482 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:03.482 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:03.483 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:17:03.483 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:07.481 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:17:09.308 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51497648, partition_space_available_1k=29592836, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=151032, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53836, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14801208, free=9994784, buffers=679496, cached=7189628, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.13, five_min_avg=3.32, fifteen_min_avg=3.84, cpu_share=2.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:17:12.486 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:17:12.486 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:17:12.486 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:12.486 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:17:12.487 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:12.487 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:12.487 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51497648 partition_space_available_1k = 29592836 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 151032 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53836 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14801208 free = 9994784 buffers = 679496 cached = 7189628 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.13 five_min_avg = 3.32 fifteen_min_avg = 3.84 cpu_share = 2.75 node_type = vrouter >> >> 10/21/2024 02:17:12.487 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:12.487 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:17:12.488 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:17:12.488 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:12.488 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:17:12.488 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:12.489 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:12.489 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:17:12.489 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:16.486 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:17:19.689 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:17:21.489 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:17:21.489 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:17:21.489 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:21.490 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:17:21.490 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:21.490 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:17:21.490 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:17:21.490 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:21.490 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:17:21.491 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:21.491 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:21.491 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:17:21.491 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:25.491 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:17:30.494 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:17:30.494 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:17:30.495 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:30.495 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:17:30.495 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:30.495 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:17:30.495 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:17:30.495 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:30.496 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:17:30.496 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:30.496 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:30.496 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:17:30.496 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:34.496 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:17:35.032 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:17:39.501 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:17:39.502 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:17:39.502 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:39.502 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:17:39.502 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:39.502 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:17:39.503 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:17:39.503 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:39.503 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:17:39.503 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:39.503 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:39.503 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:17:39.503 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:43.504 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:17:48.507 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:17:48.508 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:17:48.508 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:48.508 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:17:48.508 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:48.508 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:17:48.509 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:17:48.509 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:48.509 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:17:48.510 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:48.510 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:48.510 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:17:48.510 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:52.329 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:17:52.507 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:17:57.513 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:17:57.513 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:17:57.513 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:57.514 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:17:57.514 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:17:57.514 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:17:57.514 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:17:57.514 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:57.514 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:17:57.514 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:57.515 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:17:57.515 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:17:57.515 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:01.515 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:18:06.517 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:18:06.518 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:18:06.518 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:18:06.518 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:18:06.518 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:18:06.518 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:18:06.519 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:18:06.519 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:06.520 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:18:06.520 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:06.520 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:06.520 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:18:06.520 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:09.326 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51507780, partition_space_available_1k=29582704, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=151220, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53864, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14875672, free=9900280, buffers=679500, cached=7209664, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=4.07, five_min_avg=3.79, fifteen_min_avg=3.98, cpu_share=5.25, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:18:09.326 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:09.326 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51507780 partition_space_available_1k = 29582704 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 151220 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53864 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14875672 free = 9900280 buffers = 679500 cached = 7209664 node_type = vrouter >> system_cpu_usage = << one_min_avg = 4.07 five_min_avg = 3.79 fifteen_min_avg = 3.98 cpu_share = 5.25 node_type = vrouter >> >> 10/21/2024 02:18:09.326 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:10.517 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:18:15.523 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:18:15.523 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:18:15.523 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:18:15.523 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:18:15.523 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:18:15.523 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:18:15.524 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:18:15.524 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:15.524 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:18:15.524 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:15.524 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:15.524 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:18:15.524 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:19.525 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:18:24.531 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:18:24.531 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:18:24.531 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:18:24.531 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:18:24.531 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:18:24.531 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:18:24.532 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:18:24.532 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:24.532 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:18:24.532 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:24.532 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:24.532 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:18:24.532 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:28.533 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:18:33.535 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:18:33.536 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:18:33.536 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:18:33.536 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:18:33.536 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:18:33.536 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:18:33.537 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:18:33.537 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:33.537 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:18:33.537 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:33.537 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:33.538 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:18:33.538 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:37.539 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:18:42.541 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:18:42.542 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:18:42.542 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:18:42.542 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:18:42.542 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:18:42.542 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:18:42.542 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:18:42.542 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:42.543 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:18:42.543 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:42.543 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:42.543 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:18:42.543 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:46.545 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:18:51.549 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:18:51.549 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:18:51.549 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:18:51.549 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:18:51.549 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:18:51.549 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:18:51.550 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:18:51.550 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:51.550 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:18:51.550 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:51.550 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:51.550 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:18:51.551 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:18:55.551 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:19:00.553 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:19:00.553 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:19:00.554 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:00.554 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:19:00.554 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:00.554 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:19:00.555 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:19:00.555 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:00.555 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:19:00.555 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:00.555 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:00.555 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:19:00.556 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:04.554 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:19:09.311 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51512868, partition_space_available_1k=29577616, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=151384, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53936, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14845896, free=9919712, buffers=679616, cached=7219892, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.77, five_min_avg=3.48, fifteen_min_avg=3.86, cpu_share=2.88, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:19:09.557 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:19:09.557 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:19:09.557 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:09.557 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:19:09.557 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:09.558 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:09.558 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51512868 partition_space_available_1k = 29577616 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 151384 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53936 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14845896 free = 9919712 buffers = 679616 cached = 7219892 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.77 five_min_avg = 3.48 fifteen_min_avg = 3.86 cpu_share = 2.88 node_type = vrouter >> >> 10/21/2024 02:19:09.558 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:09.558 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:19:09.559 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:19:09.559 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:09.559 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:19:09.559 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:09.559 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:09.559 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:19:09.560 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:13.559 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:19:18.562 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:19:18.562 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:19:18.563 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:18.563 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:19:18.563 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:18.563 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:19:18.563 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:19:18.564 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:18.564 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:19:18.564 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:18.564 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:18.564 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:19:18.564 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:22.565 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:19:27.570 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:19:27.570 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:19:27.570 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:27.570 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:19:27.570 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:27.571 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:19:27.571 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:19:27.571 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:27.571 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:19:27.571 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:27.571 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:27.572 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:19:27.572 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:31.573 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:19:36.578 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:19:36.578 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:19:36.578 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:36.578 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:19:36.578 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:36.578 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:19:36.579 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:19:36.579 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:36.579 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:19:36.579 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:36.579 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:36.579 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:19:36.579 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:40.580 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:19:45.585 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:19:45.585 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:19:45.585 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:45.586 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:19:45.586 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:45.586 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:19:45.586 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:19:45.586 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:45.587 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:19:45.587 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:45.587 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:45.587 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:19:45.587 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:49.586 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:19:54.590 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:19:54.590 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:19:54.590 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:54.590 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:19:54.590 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:19:54.590 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:19:54.591 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:19:54.591 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:54.591 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:19:54.591 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:54.591 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:54.591 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:19:54.591 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:19:58.593 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:20:03.595 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:20:03.595 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:20:03.595 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:03.596 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:20:03.596 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:03.596 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:20:03.596 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:20:03.596 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:03.597 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:20:03.597 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:03.597 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:03.597 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:20:03.597 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:07.596 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:20:09.318 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51516376, partition_space_available_1k=29574108, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=151584, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53968, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14834508, free=9926544, buffers=679616, cached=7224448, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.39, five_min_avg=3.22, fifteen_min_avg=3.74, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:20:12.601 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:20:12.601 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:20:12.601 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:12.602 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:20:12.602 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:12.602 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:12.602 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51516376 partition_space_available_1k = 29574108 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 151584 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53968 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14834508 free = 9926544 buffers = 679616 cached = 7224448 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.39 five_min_avg = 3.22 fifteen_min_avg = 3.74 cpu_share = 2.62 node_type = vrouter >> >> 10/21/2024 02:20:12.602 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:12.602 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:20:12.603 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:20:12.604 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:12.604 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:20:12.604 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:12.604 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:12.604 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:20:12.604 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:16.604 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:20:21.609 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:20:21.610 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:20:21.610 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:21.610 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:20:21.610 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:21.610 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:20:21.611 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:20:21.611 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:21.611 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:20:21.611 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:21.611 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:21.611 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:20:21.611 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:25.613 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:20:30.617 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:20:30.618 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:20:30.618 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:30.618 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:20:30.618 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:30.618 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:20:30.618 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:20:30.619 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:30.619 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:20:30.619 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:30.619 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:30.619 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:20:30.619 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:34.618 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:20:39.624 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:20:39.624 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:20:39.624 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:39.624 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:20:39.624 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:39.625 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:20:39.625 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:20:39.625 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:39.625 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:20:39.625 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:39.625 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:39.625 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:20:39.626 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:43.627 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:20:48.629 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:20:48.629 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:20:48.629 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:48.629 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:20:48.630 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:48.630 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:20:48.630 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:20:48.630 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:48.630 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:20:48.630 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:48.630 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:48.631 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:20:48.631 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:52.632 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:20:57.635 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:20:57.636 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:20:57.636 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:57.636 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:20:57.636 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:20:57.636 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:20:57.637 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:20:57.637 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:57.637 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:20:57.637 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:57.637 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:20:57.637 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:20:57.637 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:01.638 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:21:06.641 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:21:06.641 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:21:06.641 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:21:06.641 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:21:06.641 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:21:06.642 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:21:06.642 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:21:06.642 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:06.643 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:21:06.643 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:06.643 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:06.643 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:21:06.643 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:09.316 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51519888, partition_space_available_1k=29570596, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=151712, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53992, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14818868, free=9937836, buffers=679620, cached=7228792, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.71, five_min_avg=3.1, fifteen_min_avg=3.66, cpu_share=2.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:21:09.317 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:09.317 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51519888 partition_space_available_1k = 29570596 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 151712 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 53992 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14818868 free = 9937836 buffers = 679620 cached = 7228792 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.71 five_min_avg = 3.1 fifteen_min_avg = 3.66 cpu_share = 2.5 node_type = vrouter >> >> 10/21/2024 02:21:09.317 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:10.642 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:21:14.671 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:21:15.644 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:21:15.645 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:21:15.645 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:21:15.645 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:21:15.645 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:21:15.645 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:21:15.645 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:21:15.645 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:15.646 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:21:15.646 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:15.646 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:15.646 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:21:15.646 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:19.645 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:21:24.649 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:21:24.649 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:21:24.649 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:21:24.649 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:21:24.649 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:21:24.649 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:21:24.650 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:21:24.650 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:24.650 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:21:24.650 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:24.650 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:24.650 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:21:24.650 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:28.650 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:21:33.654 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:21:33.654 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:21:33.654 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:21:33.654 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:21:33.655 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:21:33.655 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:21:33.655 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:21:33.655 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:33.656 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:21:33.656 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:33.656 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:33.656 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:21:33.656 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:37.657 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:21:42.663 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:21:42.663 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:21:42.663 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:21:42.663 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:21:42.663 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:21:42.663 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:21:42.664 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:21:42.664 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:42.664 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:21:42.664 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:42.664 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:42.665 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:21:42.665 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:46.665 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:21:51.668 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:21:51.668 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:21:51.669 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:21:51.669 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:21:51.669 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:21:51.669 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:21:51.670 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:21:51.670 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:51.670 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:21:51.670 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:51.670 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:51.671 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:21:51.671 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:21:55.669 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:22:00.673 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:22:00.673 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:22:00.673 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:00.674 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:22:00.674 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:00.674 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:22:00.674 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:22:00.675 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:00.675 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:22:00.675 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:00.675 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:00.675 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:22:00.675 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:04.674 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:22:09.319 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51524108, partition_space_available_1k=29566376, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=151816, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54020, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14820928, free=9930468, buffers=679812, cached=7233908, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.53, five_min_avg=2.98, fifteen_min_avg=3.59, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:22:09.676 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:22:09.676 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:22:09.676 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:09.677 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:22:09.677 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:09.677 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:09.677 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51524108 partition_space_available_1k = 29566376 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 151816 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54020 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14820928 free = 9930468 buffers = 679812 cached = 7233908 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.53 five_min_avg = 2.98 fifteen_min_avg = 3.59 cpu_share = 2.62 node_type = vrouter >> >> 10/21/2024 02:22:09.677 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:09.677 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:22:09.678 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:22:09.678 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:09.678 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:22:09.678 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:09.678 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:09.678 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:22:09.678 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:13.678 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:22:18.684 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:22:18.684 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:22:18.684 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:18.685 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:22:18.685 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:18.685 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:22:18.685 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:22:18.685 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:18.685 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:22:18.685 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:18.685 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:18.686 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:22:18.686 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:22.686 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:22:27.690 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:22:27.691 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:22:27.691 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:27.691 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:22:27.691 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:27.691 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:22:27.692 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:22:27.692 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:27.692 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:22:27.693 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:27.693 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:27.693 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:22:27.693 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:31.693 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:22:36.697 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:22:36.697 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:22:36.697 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:36.697 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:22:36.698 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:36.698 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:22:36.698 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:22:36.699 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:36.699 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:22:36.699 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:36.699 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:36.699 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:22:36.699 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:40.698 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:22:45.700 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:22:45.700 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:22:45.700 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:45.700 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:22:45.700 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:45.701 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:22:45.701 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:22:45.701 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:45.702 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:22:45.702 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:45.702 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:45.702 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:22:45.702 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:49.700 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:22:51.737 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:22:54.703 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:22:54.703 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:22:54.703 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:54.703 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:22:54.703 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:22:54.703 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:22:54.704 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:22:54.704 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:54.704 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:22:54.704 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:54.704 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:54.704 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:22:54.704 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:22:58.705 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:23:03.709 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:23:03.709 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:23:03.709 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:03.710 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:23:03.710 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:03.710 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:23:03.710 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:23:03.710 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:03.711 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:23:03.711 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:03.711 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:03.711 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:23:03.711 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:05.468 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:23:07.710 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:23:09.330 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51531976, partition_space_available_1k=29558508, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=151992, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54044, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14903108, free=9839176, buffers=679820, cached=7243012, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.41, five_min_avg=2.86, fifteen_min_avg=3.51, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:23:12.715 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:23:12.715 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:23:12.715 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:12.716 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:23:12.716 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:12.716 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:12.716 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51531976 partition_space_available_1k = 29558508 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 151992 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54044 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14903108 free = 9839176 buffers = 679820 cached = 7243012 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.41 five_min_avg = 2.86 fifteen_min_avg = 3.51 cpu_share = 2.62 node_type = vrouter >> >> 10/21/2024 02:23:12.716 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:12.716 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:23:12.717 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:23:12.717 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:12.717 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:23:12.717 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:12.717 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:12.717 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:23:12.717 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:16.719 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:23:21.721 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:23:21.722 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:23:21.722 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:21.722 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:23:21.722 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:21.722 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:23:21.723 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:23:21.723 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:21.723 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:23:21.723 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:21.723 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:21.723 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:23:21.723 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:25.723 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:23:29.307 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:23:30.725 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:23:30.725 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:23:30.725 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:30.726 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:23:30.726 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:30.726 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:23:30.726 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:23:30.726 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:30.727 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:23:30.727 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:30.727 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:30.727 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:23:30.727 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:34.727 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:23:39.730 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:23:39.730 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:23:39.731 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:39.731 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:23:39.731 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:39.731 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:23:39.731 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:23:39.731 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:39.732 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:23:39.732 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:39.732 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:39.732 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:23:39.732 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:43.733 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:23:48.737 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:23:48.737 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:23:48.737 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:48.737 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:23:48.737 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:48.738 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:23:48.738 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:23:48.738 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:48.738 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:23:48.738 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:48.738 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:48.738 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:23:48.739 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:52.738 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:23:57.743 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:23:57.743 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:23:57.743 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:57.743 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:23:57.743 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:23:57.744 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:23:57.744 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:23:57.744 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:57.744 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:23:57.744 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:57.744 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:23:57.744 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:23:57.745 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:01.745 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:24:06.747 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:24:06.747 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:24:06.747 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:24:06.748 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:24:06.748 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:24:06.748 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:24:06.749 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:24:06.749 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:06.749 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:24:06.749 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:06.749 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:06.749 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:24:06.749 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:09.318 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51538788, partition_space_available_1k=29551696, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=152124, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54076, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14867964, free=9855324, buffers=679836, cached=7261992, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=4.25, five_min_avg=3.51, fifteen_min_avg=3.7, cpu_share=5.0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:24:09.318 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:09.318 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51538788 partition_space_available_1k = 29551696 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 152124 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54076 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14867964 free = 9855324 buffers = 679836 cached = 7261992 node_type = vrouter >> system_cpu_usage = << one_min_avg = 4.25 five_min_avg = 3.51 fifteen_min_avg = 3.7 cpu_share = 5.0 node_type = vrouter >> >> 10/21/2024 02:24:09.318 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:10.747 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:24:15.752 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:24:15.753 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:24:15.753 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:24:15.753 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:24:15.753 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:24:15.754 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:24:15.754 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:24:15.754 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:15.755 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:24:15.755 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:15.755 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:15.755 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:24:15.755 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:19.756 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:24:24.758 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:24:24.758 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:24:24.759 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:24:24.759 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:24:24.759 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:24:24.759 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:24:24.760 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:24:24.760 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:24.760 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:24:24.760 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:24.760 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:24.761 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:24:24.761 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:28.761 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:24:33.766 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:24:33.766 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:24:33.766 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:24:33.767 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:24:33.767 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:24:33.767 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:24:33.768 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:24:33.768 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:33.768 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:24:33.768 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:33.768 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:33.768 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:24:33.769 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:37.766 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:24:42.770 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:24:42.771 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:24:42.771 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:24:42.771 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:24:42.771 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:24:42.771 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:24:42.772 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:24:42.772 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:42.772 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:24:42.772 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:42.772 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:42.772 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:24:42.772 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:46.773 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:24:51.774 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:24:51.775 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:24:51.775 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:24:51.775 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:24:51.775 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:24:51.775 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:24:51.776 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:24:51.777 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:51.777 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:24:51.777 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:51.777 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:51.777 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:24:51.777 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:24:55.777 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:25:00.783 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:25:00.783 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:25:00.783 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:00.783 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:25:00.783 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:00.784 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:25:00.784 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:25:00.784 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:00.784 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:25:00.784 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:00.784 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:00.785 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:25:00.785 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:04.786 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:25:09.322 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51542384, partition_space_available_1k=29548100, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=152232, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54172, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14880108, free=9833024, buffers=679848, cached=7272136, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.73, five_min_avg=3.18, fifteen_min_avg=3.57, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:25:09.788 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:25:09.788 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:25:09.788 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:09.789 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:25:09.789 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:09.789 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:09.789 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51542384 partition_space_available_1k = 29548100 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 152232 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54172 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14880108 free = 9833024 buffers = 679848 cached = 7272136 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.73 five_min_avg = 3.18 fifteen_min_avg = 3.57 cpu_share = 2.62 node_type = vrouter >> >> 10/21/2024 02:25:09.789 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:09.789 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:25:09.790 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:25:09.790 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:09.790 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:25:09.790 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:09.790 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:09.790 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:25:09.790 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:13.790 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:25:18.795 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:25:18.796 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:25:18.796 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:18.796 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:25:18.796 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:18.796 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:25:18.796 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:25:18.797 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:18.797 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:25:18.797 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:18.797 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:18.797 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:25:18.797 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:22.796 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:25:27.801 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:25:27.802 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:25:27.802 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:27.802 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:25:27.802 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:27.802 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:25:27.802 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:25:27.803 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:27.803 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:25:27.803 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:27.803 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:27.803 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:25:27.803 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:31.804 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:25:36.810 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:25:36.810 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:25:36.810 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:36.810 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:25:36.811 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:36.811 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:25:36.811 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:25:36.811 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:36.811 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:25:36.811 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:36.811 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:36.811 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:25:36.812 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:40.810 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:25:45.815 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:25:45.815 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:25:45.815 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:45.815 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:25:45.815 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:45.815 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:25:45.816 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:25:45.816 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:45.816 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:25:45.816 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:45.816 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:45.816 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:25:45.816 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:49.817 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:25:54.820 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:25:54.820 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:25:54.820 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:54.821 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:25:54.821 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:25:54.821 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:25:54.821 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:25:54.821 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:54.821 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:25:54.821 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:54.822 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:54.822 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:25:54.822 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:25:58.823 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:26:03.827 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:26:03.828 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:26:03.828 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:03.828 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:26:03.828 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:03.828 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:26:03.828 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:26:03.829 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:03.829 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:26:03.829 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:03.829 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:03.829 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:26:03.829 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:07.828 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:26:09.323 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51546188, partition_space_available_1k=29544296, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=152312, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54192, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14853116, free=9855392, buffers=679848, cached=7276760, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.71, five_min_avg=3.18, fifteen_min_avg=3.55, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:26:12.833 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:26:12.833 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:26:12.833 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:12.833 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:26:12.833 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:12.833 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:12.834 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51546188 partition_space_available_1k = 29544296 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 152312 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54192 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14853116 free = 9855392 buffers = 679848 cached = 7276760 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.71 five_min_avg = 3.18 fifteen_min_avg = 3.55 cpu_share = 2.62 node_type = vrouter >> >> 10/21/2024 02:26:12.834 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:12.834 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:26:12.834 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:26:12.834 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:12.835 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:26:12.835 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:12.835 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:12.835 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:26:12.835 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:16.836 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:26:21.841 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:26:21.841 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:26:21.841 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:21.841 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:26:21.841 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:21.841 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:26:21.842 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:26:21.842 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:21.842 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:26:21.842 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:21.842 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:21.843 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:26:21.843 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:25.843 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:26:30.848 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:26:30.848 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:26:30.849 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:30.849 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:26:30.849 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:30.849 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:26:30.849 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:26:30.850 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:30.850 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:26:30.850 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:30.850 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:30.850 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:26:30.850 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:34.851 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:26:39.854 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:26:39.854 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:26:39.854 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:39.854 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:26:39.855 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:39.855 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:26:39.855 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:26:39.856 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:39.856 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:26:39.856 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:39.856 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:39.856 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:26:39.856 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:43.857 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:26:48.863 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:26:48.863 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:26:48.863 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:48.863 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:26:48.863 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:48.863 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:26:48.864 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:26:48.864 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:48.864 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:26:48.864 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:48.864 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:48.864 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:26:48.864 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:52.863 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:26:57.869 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:26:57.869 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:26:57.869 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:57.869 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:26:57.869 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:26:57.870 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:26:57.870 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:26:57.870 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:57.870 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:26:57.870 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:57.870 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:26:57.870 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:26:57.871 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:01.870 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:27:06.871 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:27:06.872 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:27:06.872 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:27:06.872 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:27:06.872 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:27:06.872 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:27:06.874 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:27:06.874 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:06.874 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:27:06.874 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:06.874 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:06.874 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:27:06.874 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:07.429 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:27:09.326 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51552696, partition_space_available_1k=29537788, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=152424, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54252, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14921356, free=9778548, buffers=679852, cached=7285360, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=1.95, five_min_avg=2.86, fifteen_min_avg=3.42, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:27:09.326 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:09.327 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51552696 partition_space_available_1k = 29537788 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 152424 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54252 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14921356 free = 9778548 buffers = 679852 cached = 7285360 node_type = vrouter >> system_cpu_usage = << one_min_avg = 1.95 five_min_avg = 2.86 fifteen_min_avg = 3.42 cpu_share = 2.62 node_type = vrouter >> >> 10/21/2024 02:27:09.327 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:10.872 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:27:15.877 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:27:15.877 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:27:15.877 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:27:15.877 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:27:15.877 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:27:15.877 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:27:15.878 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:27:15.878 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:15.879 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:27:15.879 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:15.879 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:15.879 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:27:15.879 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:19.880 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:27:24.883 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:27:24.884 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:27:24.884 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:27:24.884 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:27:24.884 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:27:24.884 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:27:24.885 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:27:24.885 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:24.885 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:27:24.885 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:24.885 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:24.885 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:27:24.885 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:28.885 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:27:33.890 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:27:33.890 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:27:33.890 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:27:33.890 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:27:33.890 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:27:33.891 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:27:33.891 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:27:33.891 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:33.891 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:27:33.892 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:33.892 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:33.892 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:27:33.892 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:37.895 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:27:42.901 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:27:42.902 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:27:42.902 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:27:42.902 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:27:42.902 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:27:42.902 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:27:42.903 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:27:42.903 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:42.903 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:27:42.904 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:42.904 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:42.904 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:27:42.904 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:45.154 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:27:46.901 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:27:51.905 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:27:51.905 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:27:51.905 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:27:51.906 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:27:51.906 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:27:51.906 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:27:51.907 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:27:51.907 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:51.907 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:27:51.907 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:51.907 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:51.907 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:27:51.907 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:27:55.907 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:27:58.926 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:28:00.910 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:28:00.910 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:28:00.910 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:00.910 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:28:00.910 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:00.910 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:28:00.911 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:28:00.911 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:00.911 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:28:00.911 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:00.911 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:00.911 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:28:00.911 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:04.912 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:28:09.389 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51555736, partition_space_available_1k=29534748, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=152568, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54288, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=15177036, free=9516296, buffers=679852, cached=7291932, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=7.65, five_min_avg=3.92, fifteen_min_avg=3.72, cpu_share=4.0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:28:09.914 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:28:09.914 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:28:09.915 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:09.915 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:28:09.915 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:09.915 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:09.915 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51555736 partition_space_available_1k = 29534748 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 152568 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54288 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 15177036 free = 9516296 buffers = 679852 cached = 7291932 node_type = vrouter >> system_cpu_usage = << one_min_avg = 7.65 five_min_avg = 3.92 fifteen_min_avg = 3.72 cpu_share = 4.0 node_type = vrouter >> >> 10/21/2024 02:28:09.915 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:09.915 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:28:09.916 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:28:09.916 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:09.916 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:28:09.916 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:09.916 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:09.916 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:28:09.916 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:13.917 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:28:17.674 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:28:18.921 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:28:18.921 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:28:18.921 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:18.921 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:28:18.921 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:18.921 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:28:18.922 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:28:18.922 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:18.922 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:28:18.922 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:18.922 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:18.922 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:28:18.923 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:22.924 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:28:27.927 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:28:27.928 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:28:27.928 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:27.928 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:28:27.928 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:27.928 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:28:27.929 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:28:27.929 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:27.929 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:28:27.930 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:27.930 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:27.930 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:28:27.930 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:31.929 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:28:36.934 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:28:36.935 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:28:36.935 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:36.935 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:28:36.935 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:36.935 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:28:36.936 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:28:36.936 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:36.936 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:28:36.936 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:36.936 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:36.936 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:28:36.937 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:40.936 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:28:45.941 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:28:45.942 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:28:45.942 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:45.942 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:28:45.942 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:45.942 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:28:45.943 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:28:45.943 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:45.943 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:28:45.943 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:45.943 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:45.943 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:28:45.943 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:49.944 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:28:54.946 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:28:54.947 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:28:54.947 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:54.947 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:28:54.947 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:28:54.947 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:28:54.947 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:28:54.948 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:54.948 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:28:54.948 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:54.948 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:54.948 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:28:54.948 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:28:58.949 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:29:03.954 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:29:03.954 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:29:03.954 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:03.955 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:29:03.955 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:03.955 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:29:03.955 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:29:03.955 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:03.956 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:29:03.956 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:03.956 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:03.956 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:29:03.956 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:07.955 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:29:09.335 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51560908, partition_space_available_1k=29529576, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=152716, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54324, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14883528, free=9798700, buffers=679912, cached=7302976, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=4.5, five_min_avg=3.75, fifteen_min_avg=3.68, cpu_share=3.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:29:12.961 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:29:12.961 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:29:12.961 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:12.962 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:29:12.962 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:12.962 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:12.962 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51560908 partition_space_available_1k = 29529576 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 152716 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54324 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14883528 free = 9798700 buffers = 679912 cached = 7302976 node_type = vrouter >> system_cpu_usage = << one_min_avg = 4.5 five_min_avg = 3.75 fifteen_min_avg = 3.68 cpu_share = 3.75 node_type = vrouter >> >> 10/21/2024 02:29:12.962 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:12.962 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:29:12.963 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:29:12.963 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:12.964 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:29:12.964 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:12.964 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:12.964 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:29:12.964 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:16.964 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:29:21.969 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:29:21.969 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:29:21.969 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:21.969 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:29:21.969 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:21.969 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:29:21.970 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:29:21.970 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:21.970 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:29:21.970 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:21.970 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:21.970 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:29:21.970 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:25.969 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:29:30.973 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:29:30.973 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:29:30.973 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:30.973 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:29:30.973 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:30.973 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:29:30.974 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:29:30.974 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:30.974 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:29:30.974 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:30.974 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:30.975 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:29:30.975 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:34.975 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:29:39.977 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:29:39.978 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:29:39.978 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:39.978 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:29:39.978 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:39.978 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:29:39.979 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:29:39.979 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:39.979 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:29:39.979 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:39.979 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:39.979 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:29:39.979 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:43.979 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:29:48.983 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:29:48.983 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:29:48.983 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:48.983 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:29:48.983 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:48.983 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:29:48.984 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:29:48.984 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:48.984 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:29:48.984 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:48.984 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:48.984 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:29:48.984 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:52.985 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:29:57.989 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:29:57.989 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:29:57.989 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:57.989 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:29:57.989 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:29:57.989 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:29:57.990 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:29:57.990 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:57.990 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:29:57.990 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:57.990 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:29:57.991 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:29:57.991 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:01.992 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:30:06.994 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:30:06.995 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:30:06.995 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:30:06.995 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:30:06.995 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:30:06.995 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:30:06.996 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:30:06.996 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:06.996 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:30:06.996 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:06.996 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:06.996 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:30:06.997 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:09.344 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51559748, partition_space_available_1k=29530736, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=152832, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54340, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14916172, free=9765880, buffers=679912, cached=7303152, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=3.19, five_min_avg=3.51, fifteen_min_avg=3.6, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:30:09.344 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:09.345 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51559748 partition_space_available_1k = 29530736 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 152832 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54340 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14916172 free = 9765880 buffers = 679912 cached = 7303152 node_type = vrouter >> system_cpu_usage = << one_min_avg = 3.19 five_min_avg = 3.51 fifteen_min_avg = 3.6 cpu_share = 2.62 node_type = vrouter >> >> 10/21/2024 02:30:09.345 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:10.995 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:30:15.997 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:30:15.997 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:30:15.997 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:30:15.998 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:30:15.998 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:30:15.998 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:30:15.999 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:30:15.999 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:15.999 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:30:15.999 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:15.999 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:15.999 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:30:15.999 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:19.999 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:30:25.001 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:30:25.001 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:30:25.001 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:30:25.001 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:30:25.001 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:30:25.001 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:30:25.002 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:30:25.002 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:25.002 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:30:25.002 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:25.002 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:25.002 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:30:25.002 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:29.002 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:30:34.005 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:30:34.005 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:30:34.005 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:30:34.005 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:30:34.005 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:30:34.005 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:30:34.006 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:30:34.006 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:34.006 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:30:34.006 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:34.006 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:34.006 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:30:34.006 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:38.007 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:30:43.011 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:30:43.011 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:30:43.011 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:30:43.011 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:30:43.012 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:30:43.012 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:30:43.012 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:30:43.012 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:43.012 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:30:43.012 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:43.013 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:43.015 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:30:43.015 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:47.014 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:30:52.018 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:30:52.019 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:30:52.019 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:30:52.019 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:30:52.019 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:30:52.019 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:30:52.020 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:30:52.020 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:52.020 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:30:52.020 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:52.020 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:52.020 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:30:52.020 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:30:56.019 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:31:01.025 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:31:01.025 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:31:01.025 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:01.025 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:31:01.025 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:01.026 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:31:01.026 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:31:01.026 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:01.026 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:31:01.026 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:01.026 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:01.027 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:31:01.027 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:05.026 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:31:09.359 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51563584, partition_space_available_1k=29526900, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=152944, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54368, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14929340, free=9748252, buffers=679916, cached=7307608, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=3.03, five_min_avg=3.4, fifteen_min_avg=3.56, cpu_share=2.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:31:10.030 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:31:10.031 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:31:10.031 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:10.031 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:31:10.031 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:10.031 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:10.031 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51563584 partition_space_available_1k = 29526900 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 152944 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54368 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14929340 free = 9748252 buffers = 679916 cached = 7307608 node_type = vrouter >> system_cpu_usage = << one_min_avg = 3.03 five_min_avg = 3.4 fifteen_min_avg = 3.56 cpu_share = 2.75 node_type = vrouter >> >> 10/21/2024 02:31:10.031 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:10.031 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:31:10.032 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:31:10.032 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:10.032 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:31:10.032 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:10.032 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:10.032 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:31:10.032 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:14.033 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:31:19.038 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:31:19.038 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:31:19.038 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:19.038 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:31:19.038 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:19.038 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:31:19.039 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:31:19.039 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:19.040 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:31:19.040 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:19.040 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:19.040 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:31:19.040 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:23.041 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:31:28.044 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:31:28.045 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:31:28.045 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:28.045 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:31:28.045 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:28.045 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:31:28.046 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:31:28.046 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:28.046 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:31:28.046 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:28.046 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:28.046 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:31:28.046 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:32.047 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:31:37.053 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:31:37.053 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:31:37.053 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:37.054 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:31:37.054 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:37.054 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:31:37.054 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:31:37.055 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:37.055 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:31:37.055 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:37.055 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:37.055 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:31:37.055 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:41.053 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:31:44.629 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:31:46.056 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:31:46.056 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:31:46.056 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:46.056 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:31:46.056 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:46.056 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:31:46.057 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:31:46.057 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:46.057 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:31:46.057 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:46.057 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:46.057 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:31:46.057 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:50.056 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:31:55.061 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:31:55.061 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:31:55.061 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:55.062 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:31:55.062 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:31:55.062 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:31:55.062 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:31:55.063 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:55.063 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:31:55.063 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:55.063 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:55.063 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:31:55.063 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:31:59.064 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:32:04.069 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:32:04.069 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:32:04.069 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:04.069 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:32:04.070 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:04.070 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:32:04.070 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:32:04.070 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:04.071 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:32:04.071 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:04.071 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:04.071 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:32:04.071 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:08.069 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:32:09.357 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51563072, partition_space_available_1k=29527412, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=153044, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54396, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14946716, free=9730640, buffers=679916, cached=7307844, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.57, five_min_avg=3.24, fifteen_min_avg=3.49, cpu_share=2.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:32:13.072 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:32:13.072 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:32:13.072 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:13.073 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:32:13.073 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:13.073 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:13.073 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51563072 partition_space_available_1k = 29527412 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 153044 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54396 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14946716 free = 9730640 buffers = 679916 cached = 7307844 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.57 five_min_avg = 3.24 fifteen_min_avg = 3.49 cpu_share = 2.75 node_type = vrouter >> >> 10/21/2024 02:32:13.073 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:13.073 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:32:13.074 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:32:13.074 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:13.074 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:32:13.074 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:13.074 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:13.074 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:32:13.074 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:17.074 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:32:22.078 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:32:22.078 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:32:22.078 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:22.078 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:32:22.079 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:22.079 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:32:22.079 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:32:22.079 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:22.079 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:32:22.080 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:22.080 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:22.080 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:32:22.080 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:26.079 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:32:26.267 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:32:31.083 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:32:31.083 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:32:31.083 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:31.083 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:32:31.083 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:31.084 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:32:31.084 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:32:31.084 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:31.084 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:32:31.084 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:31.084 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:31.085 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:32:31.085 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:35.086 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:32:40.092 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:32:40.092 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:32:40.092 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:40.092 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:32:40.092 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:40.092 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:32:40.093 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:32:40.093 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:40.093 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:32:40.093 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:40.093 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:40.093 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:32:40.093 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:40.233 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:32:44.092 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:32:49.097 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:32:49.097 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:32:49.097 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:49.098 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:32:49.098 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:49.098 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:32:49.098 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:32:49.098 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:49.098 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:32:49.099 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:49.099 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:49.099 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:32:49.099 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:53.098 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:32:57.233 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:32:58.100 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:32:58.100 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:32:58.100 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:58.100 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:32:58.100 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:32:58.100 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:32:58.101 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:32:58.101 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:58.101 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:32:58.101 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:58.101 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:32:58.101 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:32:58.101 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:02.102 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:33:07.104 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:33:07.104 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:33:07.104 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:33:07.105 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:33:07.105 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:33:07.105 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:33:07.106 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:33:07.106 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:07.106 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:33:07.106 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:07.106 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:07.106 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:33:07.106 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:09.364 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51673652, partition_space_available_1k=29416832, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=153160, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54408, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14937144, free=9734752, buffers=679920, cached=7313300, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.34, five_min_avg=3.07, fifteen_min_avg=3.42, cpu_share=3.0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:33:09.364 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:09.365 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51673652 partition_space_available_1k = 29416832 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 153160 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54408 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14937144 free = 9734752 buffers = 679920 cached = 7313300 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.34 five_min_avg = 3.07 fifteen_min_avg = 3.42 cpu_share = 3.0 node_type = vrouter >> >> 10/21/2024 02:33:09.365 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:11.105 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:33:16.110 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:33:16.110 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:33:16.110 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:33:16.110 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:33:16.110 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:33:16.110 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:33:16.111 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:33:16.111 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:16.111 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:33:16.111 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:16.111 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:16.111 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:33:16.111 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:20.113 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:33:25.118 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:33:25.118 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:33:25.118 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:33:25.119 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:33:25.119 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:33:25.119 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:33:25.119 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:33:25.119 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:25.119 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:33:25.119 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:25.120 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:25.120 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:33:25.120 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:29.127 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:33:34.132 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:33:34.132 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:33:34.132 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:33:34.134 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:33:34.134 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:33:34.134 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:33:34.134 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:33:34.134 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:34.135 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:33:34.135 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:34.135 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:34.135 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:33:34.135 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:38.135 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:33:43.138 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:33:43.139 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:33:43.139 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:33:43.139 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:33:43.139 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:33:43.139 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:33:43.140 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:33:43.140 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:43.140 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:33:43.140 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:43.141 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:43.141 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:33:43.141 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:47.141 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:33:52.146 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:33:52.146 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:33:52.146 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:33:52.147 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:33:52.147 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:33:52.147 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:33:52.147 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:33:52.147 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:52.147 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:33:52.147 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:52.148 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:52.148 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:33:52.148 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:33:56.149 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:34:01.153 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:34:01.153 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:34:01.153 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:01.154 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:34:01.154 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:01.154 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:34:01.154 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:34:01.154 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:01.154 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:34:01.155 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:01.155 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:01.155 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:34:01.155 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:05.154 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:34:09.367 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51578044, partition_space_available_1k=29512440, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=153240, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54456, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14962440, free=9780104, buffers=679920, cached=7242652, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=5.28, five_min_avg=3.83, fifteen_min_avg=3.66, cpu_share=5.0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:34:10.156 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:34:10.157 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:34:10.157 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:10.157 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:34:10.157 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:10.157 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:10.158 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51578044 partition_space_available_1k = 29512440 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 153240 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54456 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14962440 free = 9780104 buffers = 679920 cached = 7242652 node_type = vrouter >> system_cpu_usage = << one_min_avg = 5.28 five_min_avg = 3.83 fifteen_min_avg = 3.66 cpu_share = 5.0 node_type = vrouter >> >> 10/21/2024 02:34:10.158 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:10.158 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:34:10.159 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:34:10.159 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:10.159 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:34:10.159 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:10.159 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:10.159 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:34:10.159 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:14.157 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:34:19.162 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:34:19.162 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:34:19.162 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:19.163 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:34:19.163 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:19.163 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:34:19.163 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:34:19.163 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:19.163 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:34:19.163 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:19.164 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:19.164 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:34:19.164 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:23.165 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:34:28.167 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:34:28.168 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:34:28.168 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:28.168 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:34:28.168 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:28.168 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:34:28.169 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:34:28.169 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:28.169 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:34:28.169 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:28.169 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:28.169 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:34:28.169 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:32.170 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:34:37.175 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:34:37.175 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:34:37.175 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:37.175 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:34:37.175 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:37.176 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:34:37.176 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:34:37.176 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:37.176 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:34:37.176 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:37.176 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:37.177 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:34:37.177 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:41.178 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:34:46.181 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:34:46.181 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:34:46.181 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:46.181 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:34:46.181 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:46.182 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:34:46.182 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:34:46.182 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:46.182 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:34:46.182 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:46.182 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:46.183 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:34:46.183 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:50.183 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:34:55.189 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:34:55.189 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:34:55.189 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:55.189 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:34:55.189 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:34:55.189 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:34:55.190 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:34:55.190 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:55.190 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:34:55.190 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:55.190 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:55.191 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:34:55.191 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:34:59.189 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:35:04.195 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:35:04.195 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:35:04.195 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:04.195 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:35:04.196 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:04.196 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:35:04.196 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:35:04.197 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:04.197 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:35:04.197 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:04.197 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:04.197 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:35:04.197 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:08.196 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:35:09.375 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51576920, partition_space_available_1k=29513564, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=153344, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54484, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14987880, free=9754216, buffers=679920, cached=7243100, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=3.28, five_min_avg=3.51, fifteen_min_avg=3.56, cpu_share=2.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:35:13.198 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:35:13.199 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:35:13.199 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:13.199 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:35:13.199 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:13.199 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:13.199 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51576920 partition_space_available_1k = 29513564 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 153344 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54484 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14987880 free = 9754216 buffers = 679920 cached = 7243100 node_type = vrouter >> system_cpu_usage = << one_min_avg = 3.28 five_min_avg = 3.51 fifteen_min_avg = 3.56 cpu_share = 2.75 node_type = vrouter >> >> 10/21/2024 02:35:13.200 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:13.200 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:35:13.200 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:35:13.200 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:13.201 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:35:13.201 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:13.201 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:13.201 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:35:13.201 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:17.201 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:35:22.206 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:35:22.206 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:35:22.206 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:22.206 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:35:22.206 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:22.206 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:35:22.207 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:35:22.207 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:22.207 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:35:22.207 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:22.207 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:22.207 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:35:22.207 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:26.209 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:35:31.213 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:35:31.213 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:35:31.213 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:31.213 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:35:31.213 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:31.213 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:35:31.214 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:35:31.214 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:31.214 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:35:31.214 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:31.214 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:31.214 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:35:31.215 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:35.216 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:35:40.221 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:35:40.221 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:35:40.221 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:40.221 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:35:40.221 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:40.221 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:35:40.222 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:35:40.222 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:40.222 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:35:40.222 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:40.222 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:40.222 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:35:40.222 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:44.222 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:35:49.226 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:35:49.227 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:35:49.227 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:49.227 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:35:49.227 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:49.227 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:35:49.228 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:35:49.228 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:49.228 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:35:49.228 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:49.228 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:49.228 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:35:49.228 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:53.229 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:35:58.233 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:35:58.234 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:35:58.234 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:58.234 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:35:58.234 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:35:58.234 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:35:58.235 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:35:58.235 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:58.235 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:35:58.235 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:58.235 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:58.235 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:35:58.235 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:35:58.514 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:36:02.237 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:36:07.239 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:36:07.240 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:36:07.240 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:07.240 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:36:07.240 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:07.240 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:36:07.241 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:36:07.241 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:07.241 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:36:07.241 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:07.241 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:07.242 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:36:07.242 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:09.383 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51585216, partition_space_available_1k=29505268, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=153476, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54520, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14982212, free=9751252, buffers=679924, cached=7251728, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.38, five_min_avg=3.19, fifteen_min_avg=3.44, cpu_share=2.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:36:09.383 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:09.383 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51585216 partition_space_available_1k = 29505268 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 153476 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54520 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14982212 free = 9751252 buffers = 679924 cached = 7251728 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.38 five_min_avg = 3.19 fifteen_min_avg = 3.44 cpu_share = 2.75 node_type = vrouter >> >> 10/21/2024 02:36:09.383 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:11.240 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:36:16.244 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:36:16.244 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:36:16.244 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:16.245 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:36:16.245 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:16.245 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:36:16.245 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:36:16.246 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:16.246 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:36:16.246 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:16.247 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:16.247 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:36:16.247 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:20.246 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:36:25.252 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:36:25.252 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:36:25.252 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:25.252 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:36:25.252 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:25.253 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:36:25.253 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:36:25.253 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:25.254 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:36:25.254 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:25.254 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:25.254 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:36:25.254 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:29.253 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:36:34.257 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:36:34.257 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:36:34.257 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:34.257 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:36:34.257 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:34.257 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:36:34.258 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:36:34.258 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:34.258 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:36:34.258 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:34.258 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:34.258 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:36:34.259 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:38.257 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:36:43.261 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:36:43.261 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:36:43.262 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:43.262 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:36:43.262 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:43.262 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:36:43.263 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:36:43.263 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:43.263 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:36:43.263 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:43.263 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:43.264 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:36:43.264 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:47.262 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:36:52.267 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:36:52.268 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:36:52.268 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:52.268 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:36:52.268 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:36:52.268 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:36:52.269 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:36:52.269 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:52.269 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:36:52.270 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:52.270 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:52.270 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:36:52.270 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:36:56.269 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:01.270 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:37:01.271 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:37:01.271 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:01.271 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:37:01.271 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:01.271 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:37:01.272 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:37:01.272 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:01.272 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:37:01.273 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:01.273 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:01.273 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:37:01.273 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:05.271 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:09.383 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51580624, partition_space_available_1k=29509860, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=153596, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54564, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14955604, free=9781352, buffers=679924, cached=7248236, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.14, five_min_avg=2.97, fifteen_min_avg=3.35, cpu_share=2.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:37:10.274 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:37:10.274 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:37:10.274 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:10.275 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:37:10.275 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:10.275 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:10.275 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51580624 partition_space_available_1k = 29509860 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 153596 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54564 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14955604 free = 9781352 buffers = 679924 cached = 7248236 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.14 five_min_avg = 2.97 fifteen_min_avg = 3.35 cpu_share = 2.75 node_type = vrouter >> >> 10/21/2024 02:37:10.275 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:10.275 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:37:10.276 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:37:10.276 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:10.276 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:37:10.276 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:10.277 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:10.277 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:37:10.277 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:14.279 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:19.288 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:37:19.289 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:37:19.289 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:19.289 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:37:19.289 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:19.289 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:37:19.290 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:37:19.290 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:19.290 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:37:19.290 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:19.290 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:19.290 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:37:19.291 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:23.289 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:28.293 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:37:28.293 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:37:28.293 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:28.293 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:37:28.293 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:28.294 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:37:28.294 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:37:28.294 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:28.294 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:37:28.294 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:28.295 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:28.295 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:37:28.295 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:32.295 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:37.301 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:37:37.301 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:37:37.301 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:37.302 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:37:37.302 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:37.302 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:37:37.303 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:37:37.303 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:37.303 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:37:37.303 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:37.303 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:37.303 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:37:37.304 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:41.302 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:46.304 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:37:46.304 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:37:46.304 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:46.305 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:37:46.305 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:46.305 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:37:46.305 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:37:46.305 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:46.306 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:37:46.306 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:46.306 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:46.306 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:37:46.306 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:50.307 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:55.313 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:37:55.313 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:37:55.313 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:55.314 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:37:55.314 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:37:55.314 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:37:55.315 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:37:55.315 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:55.315 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:37:55.315 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:55.315 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:55.315 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:37:55.315 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:37:59.317 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:37:59.742 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:38:04.322 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:38:04.322 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:38:04.322 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:04.323 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:38:04.323 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:04.323 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:38:04.323 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:38:04.323 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:04.323 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:38:04.323 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:04.324 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:04.324 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:38:04.324 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:08.323 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:38:09.390 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51588864, partition_space_available_1k=29501620, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=153684, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54584, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=15016828, free=9710040, buffers=679924, cached=7258324, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.06, five_min_avg=2.8, fifteen_min_avg=3.26, cpu_share=2.88, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:38:13.328 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:38:13.328 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:38:13.329 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:13.329 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:38:13.329 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:13.329 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:13.329 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51588864 partition_space_available_1k = 29501620 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 153684 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54584 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 15016828 free = 9710040 buffers = 679924 cached = 7258324 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.06 five_min_avg = 2.8 fifteen_min_avg = 3.26 cpu_share = 2.88 node_type = vrouter >> >> 10/21/2024 02:38:13.329 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:13.329 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:38:13.330 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:38:13.330 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:13.330 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:38:13.330 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:13.330 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:13.330 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:38:13.330 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:13.749 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:38:17.330 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:38:22.335 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:38:22.335 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:38:22.335 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:22.336 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:38:22.336 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:22.336 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:38:22.336 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:38:22.336 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:22.336 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:38:22.337 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:22.337 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:22.337 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:38:22.337 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:26.337 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:38:31.109 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:38:31.339 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:38:31.340 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:38:31.340 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:31.340 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:38:31.340 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:31.340 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:38:31.341 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:38:31.341 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:31.341 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:38:31.341 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:31.341 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:31.341 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:38:31.341 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:35.342 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:38:40.345 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:38:40.345 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:38:40.345 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:40.345 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:38:40.345 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:40.346 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:38:40.346 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:38:40.346 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:40.346 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:38:40.346 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:40.346 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:40.347 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:38:40.347 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:44.347 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:38:49.353 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:38:49.353 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:38:49.353 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:49.353 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:38:49.353 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:49.354 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:38:49.354 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:38:49.354 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:49.354 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:38:49.354 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:49.355 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:49.355 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:38:49.355 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:53.356 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:38:58.362 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:38:58.362 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:38:58.362 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:58.362 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:38:58.362 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:38:58.362 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:38:58.363 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:38:58.363 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:58.363 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:38:58.363 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:58.363 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:38:58.363 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:38:58.364 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:02.363 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:07.365 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:07.366 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:07.366 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:07.366 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:07.366 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:07.366 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:07.367 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:07.367 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:07.367 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:07.367 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:07.368 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:07.368 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:39:07.368 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:09.398 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51584100, partition_space_available_1k=29506384, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=153784, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54604, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=15205988, free=9523000, buffers=679924, cached=7256204, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.6, five_min_avg=2.86, fifteen_min_avg=3.25, cpu_share=3.38, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:39:09.399 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:09.399 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51584100 partition_space_available_1k = 29506384 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 153784 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54604 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 15205988 free = 9523000 buffers = 679924 cached = 7256204 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.6 five_min_avg = 2.86 fifteen_min_avg = 3.25 cpu_share = 3.38 node_type = vrouter >> >> 10/21/2024 02:39:09.399 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:11.369 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:16.371 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:16.371 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:16.371 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:16.371 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:16.371 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:16.371 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:16.372 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:16.372 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:16.372 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:16.373 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:16.373 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:16.373 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:39:16.373 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:20.373 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:25.381 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:25.381 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:25.381 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:25.382 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:25.382 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:25.382 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:25.383 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:25.383 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:25.383 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:25.383 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:25.383 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:25.383 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:39:25.383 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:29.382 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:34.385 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:34.385 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:34.385 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:34.385 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:34.385 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:34.385 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:34.386 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:34.386 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:34.386 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:34.386 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:34.386 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:34.386 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:39:34.386 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:38.387 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:43.393 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:43.394 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:43.394 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:43.394 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:43.394 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:43.394 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:43.394 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:43.395 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:43.395 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:43.395 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:43.395 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:43.395 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:39:43.395 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:47.394 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:39:52.398 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:39:52.398 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:39:52.398 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:52.398 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:39:52.399 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:39:52.399 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:39:52.399 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:39:52.399 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:52.399 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:39:52.399 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:52.399 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:52.400 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:39:52.400 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:39:56.401 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:01.406 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:01.407 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:01.407 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:01.407 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:01.407 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:01.407 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:01.408 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:01.408 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:01.408 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:01.409 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:01.411 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:01.411 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:40:01.411 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:05.409 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:09.392 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51594800, partition_space_available_1k=29495684, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=153892, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54648, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=15022980, free=9664564, buffers=681916, cached=7295656, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=3.48, five_min_avg=3.27, fifteen_min_avg=3.38, cpu_share=5.12, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:40:10.412 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:10.412 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:10.412 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:10.412 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:10.412 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:10.413 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:10.413 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51594800 partition_space_available_1k = 29495684 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 153892 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54648 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 15022980 free = 9664564 buffers = 681916 cached = 7295656 node_type = vrouter >> system_cpu_usage = << one_min_avg = 3.48 five_min_avg = 3.27 fifteen_min_avg = 3.38 cpu_share = 5.12 node_type = vrouter >> >> 10/21/2024 02:40:10.413 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:10.413 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:10.414 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:10.414 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:10.414 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:10.414 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:10.414 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:10.414 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:40:10.414 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:14.414 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:19.418 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:19.418 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:19.419 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:19.419 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:19.419 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:19.419 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:19.420 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:19.420 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:19.420 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:19.420 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:19.421 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:19.421 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:40:19.421 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:23.421 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:25.304 7f4b74379848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:40:28.425 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:28.425 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:28.426 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:28.426 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:28.426 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:28.426 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:28.426 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:28.426 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:28.427 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:28.427 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:28.427 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:28.427 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:40:28.427 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:32.426 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:37.431 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:37.431 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:37.431 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:37.431 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:37.431 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:37.431 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:37.432 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:37.432 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:37.432 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:37.432 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:37.432 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:37.432 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:40:37.432 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:41.433 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:46.437 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:46.437 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:46.437 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:46.437 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:46.437 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:46.437 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:46.438 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:46.438 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:46.438 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:46.438 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:46.438 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:46.438 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:40:46.438 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:50.437 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:40:55.442 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:40:55.442 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:40:55.442 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:55.443 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:40:55.443 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:40:55.443 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:40:55.443 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:40:55.443 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:55.443 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:40:55.444 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:55.444 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:55.444 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:40:55.444 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:40:59.444 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:04.446 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:04.446 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:04.446 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:04.446 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:04.446 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:04.447 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:04.447 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:04.447 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:04.447 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:04.447 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:04.447 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:04.447 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:41:04.448 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:08.447 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:09.401 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51599772, partition_space_available_1k=29490712, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=153992, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54684, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=15013444, free=9668524, buffers=681916, cached=7301232, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.21, five_min_avg=2.97, fifteen_min_avg=3.27, cpu_share=2.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:41:13.452 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:13.452 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:13.453 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:13.453 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:13.453 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:13.453 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:13.454 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51599772 partition_space_available_1k = 29490712 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 153992 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54684 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 15013444 free = 9668524 buffers = 681916 cached = 7301232 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.21 five_min_avg = 2.97 fifteen_min_avg = 3.27 cpu_share = 2.75 node_type = vrouter >> >> 10/21/2024 02:41:13.454 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:13.454 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:13.454 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:13.454 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:13.455 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:13.455 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:13.455 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:13.455 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:41:13.455 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:17.453 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:22.460 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:22.460 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:22.460 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:22.460 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:22.460 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:22.460 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:22.461 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:22.461 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:22.461 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:22.461 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:22.461 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:22.462 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:41:22.462 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:26.461 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:31.465 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:31.465 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:31.465 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:31.465 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:31.465 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:31.465 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:31.466 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:31.466 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:31.466 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:31.466 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:31.466 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:31.466 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:41:31.466 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:35.467 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:40.473 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:40.473 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:40.473 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:40.473 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:40.474 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:40.474 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:40.474 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:40.475 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:40.475 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:40.475 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:40.475 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:40.475 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:41:40.475 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:44.473 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:49.476 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:49.476 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:49.476 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:49.476 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:49.476 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:49.476 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:49.477 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:49.477 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:49.477 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:49.477 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:49.477 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:49.478 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:41:49.478 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:53.479 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:41:58.484 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:41:58.484 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:41:58.484 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:58.485 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:41:58.485 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:41:58.485 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:41:58.485 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:41:58.485 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:58.486 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:41:58.486 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:58.486 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:41:58.486 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:41:58.486 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:02.487 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:07.490 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:07.490 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:07.491 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:07.491 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:07.491 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:07.491 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:07.492 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:07.492 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:07.493 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:07.493 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:07.493 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:07.493 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:42:07.493 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:09.394 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51598512, partition_space_available_1k=29491972, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=154084, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54700, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14993464, free=9689184, buffers=681916, cached=7300552, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.28, five_min_avg=2.86, fifteen_min_avg=3.22, cpu_share=2.88, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:42:09.394 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:09.395 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51598512 partition_space_available_1k = 29491972 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 154084 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54700 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14993464 free = 9689184 buffers = 681916 cached = 7300552 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.28 five_min_avg = 2.86 fifteen_min_avg = 3.22 cpu_share = 2.88 node_type = vrouter >> >> 10/21/2024 02:42:09.395 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:11.490 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:16.495 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:16.495 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:16.496 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:16.496 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:16.496 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:16.496 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:16.497 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:16.497 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:16.497 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:16.497 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:16.497 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:16.497 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:42:16.497 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:20.496 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:25.502 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:25.502 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:25.502 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:25.502 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:25.503 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:25.503 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:25.503 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:25.503 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:25.504 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:25.504 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:25.504 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:25.504 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:42:25.504 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:29.503 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:34.506 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:34.506 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:34.506 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:34.506 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:34.506 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:34.506 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:34.507 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:34.507 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:34.507 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:34.507 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:34.507 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:34.507 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:42:34.507 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:38.509 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:43.513 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:43.513 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:43.513 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:43.513 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:43.514 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:43.514 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:43.514 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:43.514 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:43.514 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:43.514 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:43.515 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:43.515 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:42:43.515 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:47.516 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:42:52.521 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:42:52.521 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:42:52.521 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:52.522 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:42:52.522 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:42:52.522 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:42:52.522 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:42:52.522 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:52.523 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:42:52.523 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:52.523 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:52.523 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:42:52.523 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:42:56.524 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:01.530 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:01.530 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:01.530 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:01.530 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:01.530 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:01.531 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:01.531 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:01.531 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:01.531 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:01.531 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:01.531 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:01.532 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:43:01.532 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:05.531 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:09.390 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51602512, partition_space_available_1k=29487972, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=154188, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54732, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=14975128, free=9702768, buffers=681916, cached=7305304, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.35, five_min_avg=2.76, fifteen_min_avg=3.16, cpu_share=2.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 10/21/2024 02:43:10.533 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:10.533 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:10.534 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:10.534 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:10.534 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:10.534 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:10.534 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51602512 partition_space_available_1k = 29487972 percentage_partition_space_used = 64 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 154188 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 54732 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665116 used = 14975128 free = 9702768 buffers = 681916 cached = 7305304 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.35 five_min_avg = 2.76 fifteen_min_avg = 3.16 cpu_share = 2.75 node_type = vrouter >> >> 10/21/2024 02:43:10.534 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:10.534 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:10.536 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:10.536 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:10.536 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:10.536 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:10.536 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:10.536 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:43:10.536 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:14.535 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:19.537 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:19.537 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:19.537 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:19.537 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:19.537 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:19.537 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:19.538 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:19.538 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:19.538 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:19.538 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:19.538 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:19.539 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:43:19.539 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:23.540 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:28.545 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:28.545 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:28.545 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:28.545 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:28.545 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:28.545 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:28.546 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:28.546 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:28.546 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:28.546 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:28.546 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:28.546 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:43:28.546 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:32.547 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:37.550 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:37.550 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:37.550 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:37.550 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:37.550 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:37.550 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:37.551 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:37.551 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:37.551 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:37.551 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:37.551 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:37.551 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:43:37.551 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:41.553 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:46.557 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:46.557 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:46.557 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:46.558 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:46.558 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:46.558 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:46.558 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:46.558 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:46.559 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:46.559 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:46.559 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:46.559 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:43:46.559 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:50.558 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:43:55.562 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:43:55.562 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:43:55.562 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:55.562 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:43:55.562 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:43:55.563 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:43:55.563 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:43:55.563 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:55.563 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:43:55.563 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:55.564 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:55.564 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:43:55.564 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:43:56.168 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:43:57.170 7f4b74273848 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 10/21/2024 02:43:59.563 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:44:04.565 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/21/2024 02:44:04.565 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/21/2024 02:44:04.566 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:44:04.566 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/21/2024 02:44:04.566 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/21/2024 02:44:04.566 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/21/2024 02:44:04.566 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/21/2024 02:44:04.566 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:44:04.567 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-439-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/21/2024 02:44:04.567 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:44:04.567 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:44:04.567 7f4b74273848 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-439-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1729476185905572 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 10/21/2024 02:44:04.567 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/21/2024 02:44:08.566 7f4b74273848 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/21/2024 02:44:09.395 7f4b7a965f20 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-439-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51709892, partition_space_available_1k=29380592, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=154316, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54764, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=15076096, free=9338928, buffers=684684, cached=7565408, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.85, five_min_avg=2.77, fifteen_min_avg=3.13, cpu_share=3.12, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter')