05/08/2025 01:54:21.964 7f33bae170f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Cannot write http_port 8113 to /tmp/contrail-analytics-alarm-nodemgr.0.http_port
05/08/2025 01:54:21.965 7f33bae170f0 [contrail-analytics-alarm-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8113
05/08/2025 01:54:21.967 7f33bae15a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-186-1.cluster.local  process_status = [ <<  module_id = contrail-analytics-alarm-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/08/2025 01:54:21.968 7f33bae15a20 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-186-1.cluster.local:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8113  start_time = 1746669261967401  collector_name =   collector_ip =   collector_list = [ 10.0.0.49:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 1  >>  
05/08/2025 01:54:22.414 7f33bae2fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-186-1.cluster.local  build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-08 00:51:57.996615", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9112-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}  system_cpu_info = <<  num_socket = 1  num_cpu = 4  num_core_per_socket = 4  num_thread_per_core = 1  >>  running_package_version = nightly-nightly.el7  installed_package_version = nightly-nightly.el7  >>  
05/08/2025 01:54:22.414 7f33bae2fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-186-1.cluster.local  process_status = [ <<  module_id = contrail-analytics-alarm-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/08/2025 01:54:22.415 7f33bae2fcc0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-186-1.cluster.local  deleted = False  process_info = [ <<  process_name = contrail-alarm-gen  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1746669259000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = kafka  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1746669234000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-analytics-alarm-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1746669259000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-08 00:51:57.996615", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9112-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}  >>  
05/08/2025 01:54:22.419 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 01:54:25.969 7f33b8e1c8d0 [contrail-analytics-alarm-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/08/2025 01:54:25.969 7f33b8e1c8d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-186-1.cluster.local  process_status = [ <<  module_id = contrail-analytics-alarm-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.49:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/08/2025 01:54:25.970 7f33b8e1c8d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-186-1.cluster.local  process_status = [ <<  module_id = contrail-analytics-alarm-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.49:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/08/2025 01:54:25.970 7f33b8e1c8d0 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-186-1.cluster.local:AnalyticsAlarm:contrail-analytics-alarm-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8113  start_time = 1746669261967401  collector_name =   collector_ip = 10.0.0.49:8086  collector_list = [ 10.0.0.49:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 3  >>  
05/08/2025 01:54:27.525 7f33b8e1c438 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-186-1.cluster.local  disk_usage_info = { /dev/vda5 : <<  partition_type = xfs  partition_space_used_1k = 10563380  partition_space_available_1k = 72179896  percentage_partition_space_used = 13  >> }  process_mem_cpu_usage = { contrail-alarm-gen : <<  mem_virt = 1744  cpu_share = 0  mem_res = 0  >>kafka : <<  mem_virt = 487784  cpu_share = 0  mem_res = 0  >>contrail-analytics-alarm-nodemgr : <<  mem_virt = 49300  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16115472  used = 4306956  free = 2270004  buffers = 5196  cached = 9533316  node_type = analytics-alarm-node  >>  system_cpu_usage = <<  one_min_avg = 26.18  five_min_avg = 6.28  fifteen_min_avg = 2.16  cpu_share = 0  node_type = analytics-alarm-node  >>  >>  
05/08/2025 01:54:29.978 7f33b6c9b390 [contrail-analytics-alarm-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-186-1.cluster.local  process_status = [ <<  module_id = contrail-analytics-alarm-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.49:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/08/2025 01:55:22.528 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 01:56:22.437 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 01:57:22.439 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 01:58:22.446 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 01:59:22.468 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:00:22.458 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:01:22.462 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:02:22.469 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:03:22.471 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:04:22.493 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:05:22.475 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:06:22.497 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:07:22.500 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:08:22.513 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:09:22.541 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:10:22.537 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:11:22.549 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:12:22.555 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:13:22.563 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:14:22.590 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:15:22.578 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:16:22.585 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:17:22.587 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:18:22.590 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:19:22.613 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:20:22.595 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/08/2025 02:21:22.598 7f33bb3ca788 [contrail-analytics-alarm-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'