05/15/2025 01:39:26.051 7f1086c080f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Cannot write http_port 8113 to /tmp/contrail-analytics-alarm-nodemgr.0.http_port
05/15/2025 01:39:26.052 7f1086c080f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8113
05/15/2025 01:39:26.054 7f1086c14a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-188-1.cluster.local process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >>
05/15/2025 01:39:26.055 7f1086c14a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-k8s-manifests-188-1.cluster.local:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1747273166054020 collector_name = collector_ip = collector_list = [ 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >>
05/15/2025 01:39:27.487 7f108d411a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-188-1.cluster.local build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-15 00:43:29.193890", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9180-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >>
05/15/2025 01:39:27.494 7f108d411a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-188-1.cluster.local process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >>
05/15/2025 01:39:27.494 7f108d411a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-188-1.cluster.local deleted = False process_info = [ << process_name = contrail-alarm-gen process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1747273160000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = kafka process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1747273134000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-analytics-alarm-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1747273160000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-15 00:43:29.193890", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9180-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >>
05/15/2025 01:39:27.515 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:39:30.060 7f1084f85390 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/15/2025 01:39:30.061 7f1084f85390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-188-1.cluster.local process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >>
05/15/2025 01:39:30.061 7f1084f85390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-188-1.cluster.local process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >>
05/15/2025 01:39:30.061 7f1084f85390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-k8s-manifests-188-1.cluster.local:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1747273166054020 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >>
05/15/2025 01:39:32.567 7f108e0580f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-188-1.cluster.local disk_usage_info = { /dev/vda5 : << partition_type = xfs partition_space_used_1k = 10800616 partition_space_available_1k = 71942660 percentage_partition_space_used = 13 >> } process_mem_cpu_usage = { contrail-alarm-gen : << mem_virt = 1740 cpu_share = 0 mem_res = 0 >>kafka : << mem_virt = 495536 cpu_share = 0 mem_res = 0 >>contrail-analytics-alarm-nodemgr : << mem_virt = 49396 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16115472 used = 5812332 free = 700388 buffers = 5196 cached = 9597556 node_type = analytics-alarm-node >> system_cpu_usage = << one_min_avg = 11.75 five_min_avg = 5.39 fifteen_min_avg = 2.06 cpu_share = 0 node_type = analytics-alarm-node >> >>
05/15/2025 01:39:34.088 7f108e0580f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/15/2025 01:39:34.088 7f108e0580f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-188-1.cluster.local process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >>
05/15/2025 01:39:34.089 7f108e0580f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-188-1.cluster.local process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >>
05/15/2025 01:39:34.089 7f108e0580f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-k8s-manifests-188-1.cluster.local:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8113 start_time = 1747273166054020 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >>
05/15/2025 01:39:38.096 7f108e0580f0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-188-1.cluster.local process_status = [ << module_id = contrail-analytics-alarm-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >>
05/15/2025 01:40:27.543 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:41:27.508 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:42:27.510 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:43:27.513 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:44:27.531 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:45:27.516 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:46:27.531 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:47:27.546 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:48:27.551 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:49:27.578 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:50:27.565 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:51:27.574 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:52:27.587 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:53:27.588 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:54:27.608 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:55:27.598 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:56:27.600 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:57:27.600 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:58:27.600 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 01:59:27.648 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:00:27.628 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:01:27.641 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:02:27.653 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:03:27.662 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:04:27.703 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:05:27.680 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:06:27.689 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:07:27.694 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:08:27.696 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:09:27.713 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:10:27.701 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:11:27.707 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:12:27.722 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:13:27.753 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:14:27.756 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:15:27.753 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:16:27.766 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:17:27.772 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:18:27.780 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:19:27.800 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/15/2025 02:20:27.794 7f1087280c88 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'